testing tutorials

Security Testing Tutorial

Rahul — Sat, 28 Oct 2023 04:05:33 +0000

What is Security Testing?

Security testing is a process of evaluating a software application, system, or network to identify and rectify vulnerabilities and weaknesses that could lead to unauthorized access, data breaches, or system compromises.

Why is Security Testing Important?

Security testing is crucial because it helps:

Protect sensitive data
Ensure the confidentiality, integrity, and availability of the system
Comply with industry standards and regulations
Prevent financial loss and damage to reputation

Types of Security Testing

Penetration Testing

Penetration testing, often referred to as “ethical hacking,” involves simulating real attacks to identify vulnerabilities. Skilled testers attempt to exploit weaknesses to demonstrate potential risks.

Vulnerability Scanning

Vulnerability scanning uses automated tools to identify known vulnerabilities in your system or network. It’s an efficient way to detect low-hanging fruit.

Security Auditing

Security auditing involves reviewing system configurations, access controls, and policies to identify security weaknesses. This can be done manually or with the help of tools.

Security Code Review

Security code review involves examining the source code for security flaws, such as injection vulnerabilities, insecure data storage, and authentication issues. Tools like static code analyzers can help.

Threat Modeling

Threat modeling is a proactive approach to security. It identifies potential threats and vulnerabilities during the design phase, allowing you to mitigate them early.

Security Testing Process

Planning

Define the scope, objectives, and methodologies of your security testing. Establish timelines and gather the necessary resources.

Scanning

Perform security scans and tests using the selected tools and methodologies. This may include penetration testing, vulnerability scanning, and code reviews.

Analysis

Examine the results of your tests, prioritize vulnerabilities, and assess their potential impact. Determine whether they are false positives or actual threats.

Reporting

Create a comprehensive report detailing the vulnerabilities, their severity, and recommendations for remediation. Share this with the development and operations teams.

Remediation

Collaborate with developers and administrators to fix identified vulnerabilities. Ensure proper testing and validation of fixes before deploying updates.

Security Testing Best Practices

Define a Security Testing Strategy

Establish clear objectives, scope, and methodologies for your security testing. Consider the critical areas to focus on based on your application’s functionality.

Understand Your Application

Comprehend the architecture, components, and data flows in your application. Knowing how your system works is essential for identifying potential vulnerabilities.

Create Test Cases

Develop comprehensive test cases that cover different aspects of security, such as authentication, authorization, data validation, and encryption.

Use Security Testing Tools

Leverage security testing tools to automate scans and assessments. Some popular tools include OWASP ZAP, Nessus, Burp Suite, Metasploit, and Wireshark.

Test for OWASP Top 10 Vulnerabilities

Pay special attention to the OWASP Top 10, a list of the most critical web application security risks. These include issues like injection attacks, broken authentication, and security misconfigurations.

Conduct Regular Testing

Security is an ongoing process. Schedule regular security tests to ensure that new vulnerabilities are not introduced as your application evolves.

How to perform security testing parallel in each stage of the software development life cycle(SDLC)

Performing security testing in parallel with each stage of the Software Development Life Cycle (SDLC) is a best practice known as “Shift Left” security. This approach ensures that security is considered and integrated from the early stages of development rather than being tacked on at the end. Here’s how you can perform security testing at each stage of the SDLC:

Requirements Phase:
Threat Modeling: Identify potential security threats and vulnerabilities based on project requirements. Use threat modeling tools to visualize and analyze the threat landscape.
Design Phase:
Security Architecture Review: Assess the security of the system’s architecture and design. Ensure that proper security measures, such as authentication, authorization, and encryption, are incorporated into the design.
Development Phase:
– Static Application Security Testing (SAST): Use SAST tools to analyze the source code for security vulnerabilities. Developers can receive early feedback on issues they introduce.
– Code Review: Conduct manual code reviews to identify security flaws, such as injection vulnerabilities, authentication issues, and insecure data handling.
Testing Phase:
– Dynamic Application Security Testing (DAST): Perform dynamic scanning of the running application to find vulnerabilities like input validation issues, session management problems, and other runtime issues.
– Penetration Testing: Simulate real-world attacks to identify vulnerabilities that automated testing might miss. This stage helps validate the effectiveness of security controls.
Deployment Phase:
Security Configuration Audit: Ensure that the deployed system is configured securely. This includes reviewing server configurations, database settings, and access controls.
Operations and Maintenance Phase:
– Continuous Monitoring: Implement tools for continuous security monitoring to detect and respond to new vulnerabilities and threats as they emerge.
– Incident Response: Develop and regularly update an incident response plan to react to security incidents promptly.
Post-Release Phase:
Security Patch Management: Continuously monitor for security updates and apply patches to address vulnerabilities as they are discovered.
User Training and Awareness
Educate end-users and administrators about security best practices and the importance of maintaining a secure environment.

To perform security testing effectively in parallel with each stage of the SDLC:
Integrate Security into Workflows: Incorporate security checks and approvals into your development and release pipelines. Automated tools, like CI/CD security plugins, can help with this.
Collaboration: Promote collaboration between security teams and development teams. Security experts should work closely with developers to provide guidance and support.
Security Standards and Guidelines: Establish security coding standards and guidelines that developers should follow throughout the development process.
Automate Security Testing: Utilize automated security testing tools to ensure consistency and efficiency in identifying vulnerabilities.
Documentation and Reporting: Maintain detailed documentation of security findings and regularly communicate security status to relevant stakeholders.
Regular Training: Provide ongoing training and awareness programs for development and operations teams to keep them updated on the latest security threats and best practices.

Example Test Scenarios for Security Testing

Here are some example test scenarios for security testing, organized by various aspects of security to cover a wide range of potential vulnerabilities and threats:

1. Authentication and Authorization:

1.1. Authentication Bypass:

Verify that an unauthenticated user cannot access restricted areas or features.
Attempt to log in with incorrect credentials and check if the system prevents unauthorized access.

1.2. Password Policy:

Test the system to ensure it enforces password complexity rules.
Verify that users are prompted to change their passwords periodically.

1.3. Session Management:

Log in with one account, and then attempt to access another user’s account to ensure proper session isolation.
Check for session timeouts and session fixation vulnerabilities.

2. Input Validation:

2.1. SQL Injection:

Attempt to inject SQL code into input fields to verify that the system prevents SQL injection attacks.

2.2. Cross-Site Scripting (XSS):

Try injecting malicious scripts into user inputs to check if the application escapes or filters out dangerous script code.

2.3. Cross-Site Request Forgery (CSRF):

Craft a request that tricks an authenticated user into executing unwanted actions, such as changing their password or sending funds.

3. Data Security:

3.1. Data Encryption:

Verify that sensitive data, such as passwords and payment information, are properly encrypted during transmission (using HTTPS) and storage.

3.2. Data Leakage:

Attempt to access restricted data by manipulating URLs, cookies, or hidden fields.
Verify that confidential data is not exposed in error messages.

4. Configuration Management:

4.1. Security Misconfiguration:

Check for default configurations and ensure they have been changed to more secure settings.
Verify that unnecessary services and ports are not exposed.

4.2. File Uploads:

Attempt to upload malicious files and verify that the system correctly identifies and blocks them.
Ensure that uploaded files are stored securely and cannot be executed on the server.

5. Access Control:

5.1. Unauthorized Access:

Check if regular users can access administrative functions.
Verify that users can only view and modify data they are authorized to access.

5.2. Role-Based Access Control:

Confirm that different roles have the appropriate permissions, and users cannot escalate their privileges.

6. Error Handling:

6.1. Detailed Error Messages:

Attempt to cause errors in the application and verify that detailed error messages do not reveal sensitive information about the system.

7. Business Logic:

7.1. Business Logic Flaws:

Test the application for logical vulnerabilities, such as order manipulation, price changes, or access to unauthorized resources.

8. API Security:

8.1. API Authentication and Authorization:

Ensure that APIs are properly protected with authentication and authorization mechanisms.
Test for insecure direct object references.

8.2. API Rate Limiting and Throttling:

Verify that APIs have rate limiting to prevent abuse.

Security Testing Tools

Web Application Security Testing:

OWASP ZAP (Zed Attack Proxy): An open-source web application security scanner and proxy tool. It helps you find security vulnerabilities in web applications during development and testing.
Burp Suite: A comprehensive tool for web application security testing, including scanning, crawling, and proxy features. It is widely used by penetration testers and security professionals.
Nessus: A powerful vulnerability scanner that can scan networks for security vulnerabilities and misconfigurations. It provides detailed reports and recommendations for remediation.
Acunetix: A web vulnerability scanner that can detect a wide range of security issues in web applications, including SQL injection, cross-site scripting, and more.
Nikto: An open-source web server scanner that identifies various vulnerabilities and misconfigurations, such as outdated software and security issues.
AppScan (IBM Security AppScan): A tool for web application security testing, offering features like dynamic scanning, static scanning, and interactive scanning.

Network Security Testing:

Wireshark: A widely-used network protocol analyzer that helps you capture and inspect data on a network. It is invaluable for understanding network traffic and diagnosing network issues.
Nmap (Network Mapper): A versatile network scanning tool that can discover open ports, services, and vulnerabilities on remote hosts.
Metasploit: A penetration testing framework that includes a collection of tools for finding and exploiting vulnerabilities. It is often used by security professionals for ethical hacking.

Code Analysis and Static Analysis:

Checkmarx: A static code analysis tool that identifies security vulnerabilities in source code. It’s used to find and remediate security issues early in the development process.
Fortify (Micro Focus Fortify): A suite of tools for static and dynamic application security testing. It helps identify and fix vulnerabilities in code.

Mobile Application Security Testing:

MobSF (Mobile Security Framework): An open-source mobile application security assessment tool that supports Android and iOS applications.
Drozer: A comprehensive Android security testing framework that helps find and exploit vulnerabilities in Android apps.

Cloud Security Testing:

Prowler: A security tool for AWS that performs security best practices checks on your AWS infrastructure.
Terraform Compliance: A tool to check security compliance with Terraform templates, which are used for infrastructure as code in cloud environments.

Database Security Testing:

Sqlmap: An open-source penetration testing tool that automates the process of detecting and exploiting SQL injection vulnerabilities.
DbProtect (Trustwave DbProtect): A database security tool that identifies vulnerabilities and misconfigurations in databases and provides recommendations for securing them.

Operating System and Server Security Testing:

OpenVAS (Open Vulnerability Assessment System): An open-source vulnerability scanner that can scan for vulnerabilities on servers and systems.
Lynis: A security auditing tool for Unix and Linux-based systems. It performs system hardening and compliance testing.

Security testing is an integral part of ensuring the safety and integrity of your software and systems. By following best practices, using appropriate tools, and conducting various types of security testing, you can significantly reduce the risk of security breaches and protect your organization’s valuable assets. Remember that security is an ongoing process, so regularly test and update your security measures to stay ahead of potential threats.

20 API Testing Tools – Beyond SOAP UI and Postman

Rahul — Fri, 10 Apr 2020 01:36:06 +0000

We all know about API testing well. In my previous article – API testing, Key Terminologies and more……we learnt about the basics of API testing. Let’s now take a look on some of the API Testing tools.

To learn something new, I haven’t considered the leaders like SOAP UI, Postman, JMeter. Rather I have explored some tools which may be lesser known but have more or less same features and give a solid performance therefore worth giving a shot.

#1 ACCELQ

Zero code API Test Automation on Cloud.
Support for leading technologies.
API and UI Test Automation in the same simplified flow.
CI Driven regression suite executions.
Dynamic Environment management.
Directly correlate Business process and corresponding API for complete coverage.
API Test Case Management, Test Planning, Execution and tracking governance.
Trial – 14 days free trial

#2 API Fortress

API Fortress is a REST & SOAP API, Web Services, and Microservices testing platform.
Out-of-the-Box Integrations.
Deploy On-Premises or in the Cloud.
Quickly Generate and Edit Functional Tests
Detailed Reporting.
30 days free trial

#3 API Science

API Science is an advanced API Monitoring service.
Monitor own APIs
Multi-step REST transactions
Advanced tests & validations
Get alerted on any issue
Run tests at any time
Monitor your own API & 3rd party APIs
Customized reporting
Intelligent API diagnostics
Collaborate using
team plans

#4 Apigee

Easily create API proxies and visually configure or code API policies as steps in the API flow.
Enforce consistent security best practices and governance policies across all APIs.
Publish APIs by providing reference documentation, manage the audience for an API from portal.
Analyze APIs, API traffic data, investigate traffic spikes, and trace live API calls.
Monitor APIs to ensure API availability.
Publish APIs to a customizable portal, so that developers can easily explore, test, get API keys, and innovate fast.
60 days free trial.

#5 Assertible

Can automatically sync API tests with the latest changes in the specification.
Integrates with the tools e.g. push code to GitHub or send alerts to Slack if failures happen.
Easily create tests for web service.
Continuously test multiple deployment and environment.
Set up scheduled monitoring to continuously test the services.
Run the same tests on multiple environments.
Automatically execute post-deploy smoke tests.
HTTP assertions allow you to create domain specific test.

#6 Citrus Framework

Citrus is open source and licensed under Apache License 2.0.
Test automation by creating repeatable integration tests within continuous build.
Test complex messaging scenarios with step by step sequence of operations.
Use advanced logic in test cases including loops, delays, retries, parallel sections.
Json message payload validation.
JsonPath element validation.
Groovy Json validation.
Simple test creation.
IDE support – execute tests as TestNG/JUnit tests from Eclipse, IntelliJ IDEA or NetBeans.

#7 HTTPMASTER

Development and test tool for REST web services and API applications.
Can display and validate the most common REST formats; XML, JSON, and HTML
Broad set of http properties.
Response data validation
Simulate variations of input data or create global values for reuse across multiple API calls.
Enables to define chaining items of different types.
Support for data upload which enables users to post plain text data, contents of file, or mix of plain text data and disk files.
Free 30-day trial

#8 Insomnia

Free and open source on Mac, Windows, and Linux.
Reuse API keys or session IDs. Define environment variables globally or switch between sub-environments.
Generate http code for over thirty language libraries, including Curl, NodeJS, Go, Swift, Python, Java, C.
Free version available.

#9 Mockbin

Mockbin allows you to generate custom endpoints to test, mock, and track HTTP requests & responses between libraries, sockets and APIs.
Mock custom endpoints using any HTTP Archive (HAR) response object.
Test using your preferred language.
Mockbin accepts all standard Methods and allows method overriding in addition to Get and Post.
Mockbin supports output in JSON, YAML and XML, as well as an HTML.
Log and inspect incoming calls to your custom endpoints.

#10 Ping API

Ping-API is API testing allows to write test script in JavaScript and CoffeeScript.
It allows inspecting HTTP API call with a complete request and response data.
Can schedule test in every minutes or hour
Support for writing script to set request headers, body and URL parameters.
It also supports validation of response headers and body.
Easy to generate test script with the input of API parameters.
Free for limited projects, varied pricing afterwards.

#11 Postwoman

Postwoman is an open-source API request builder. It helps you create your requests faster, saving you precious time on your development.

It also helps to create the requests faster, saving precious time on overall development.

Free, Open Sourced
Runs online
Self Hosted
Multi-platform & multi-device support
Progressive Web App
WebSocket testing
Customizations allowed
Pre-Request Scripts and environment variables setup
Docker support

#12 pyresttest

A REST testing and API microbenchmarking tool.
Tests are defined in basic YAML or JSON config files, no code needed.
Minimal dependencies (pycurl, pyyaml, optionally future), making it easy to deploy on-server for smoketests/healthchecks.
Supports generate/extract/validate mechanisms to create full test scenarios.
Returns exit codes on failure, to slot into automated configuration management/orchestration tools (also supplies parseable logs).
Logic is written and extensible in Python.
Return exit codes for failed results.
The building of test scenarios with generate/extract/validates mechanisms.
Easy deployment on the server.

#13 ReadyAPI

ReadyAPI is an integrated suite of API testing tools that includes:

SoapUI Pro (functional testing)
LoadUI Pro (performance testing)
ServiceV Pro (service virtualization)

Metrics & Data In One Place.
Create and maintain all the information relating to projects including test cases, data sources, and object components in one place for re-usability and management.
The entire ReadyAPI suite uses a central, standardized reporting function to support the measuring and displaying of data and metrics in real time.
Continuous Integration with ReadyAPI.
ReadyAPI provides many options for scripting to create functional, load, or security tests, using either Groovy or JavaScript.
With API Discovery, ReadyAPI will listen to live traffic of an API, recording responses and different interactions captured during the API activity.

#14 RestSharp

RestSharp is a popular HTTP client library for .NET.
Featuring automatic serialization and deserialization, request and response type detection, variety of authentications.
JSON, XML and custom serialization and deserialization.
Basic, OAuth 1, OAuth 2, JWT, NTLM are supported.
HTTP call can be customized.
Send objects as the request body in JSON or XML, or as a form.

#15 SOAPSonar

SOAPSonar provides automated testing of each API independent of client, device or Consumer.
Automatically generating a UI to provide immediate feedback to developers, testers.
Validates each API independently, shortens the critical path and supports Agile development.
Fully automated, requires no scripting or maintenance thus reducing time and skill-set required.
Integration with variety test management software hence provides detailed progress reports.
Regression testing can be automated, providing early identification of changes to the API or versions.
Support for message and protocol layer identity standards allows for cloud testing.

#16 SOAtest

Broadest support for over 100 message formats and protocols.
With visual drag-and-drop tooling, users can create the most complex test scenarios without having to write a single line of code.
Test flow logic, complex assertions, looping, data driving, and keyword association, such as BDD with Cucumber, can all be easily built with minimal technical experience.
With its SMART API Test Generator plugin for Chrome, SOAtest monitors activity in your web UI from manual or exploratory testing and converts the behind-the-scenes API calls into meaningful API test scenarios.
Integration with existing CI/CD systems (i.e. Jenkins, TeamCity, Bamboo, Azure DevOps).
Collects REST and SOAP API coverage as part of test execution, reporting the test case coverage, missing or incomplete.

#17 TestMace

Creating and testing complex scenarios quickly.
Creating tests without actual programming.
Powerful autocomplete feature and highlighting of current variables values, functions etc.
A well-organized project structure and human-readable file format, which allow to store your project in the version control system repo and review tests and scenarios.
Advanced JS support in the built-in code editor, including the autocomplete feature and static analyzer.

#18 vRest

Record test cases by manually using your application once.
vrunner lets you execute your test cases just by running a command on your terminal.
Integration with Continuous Integration servers like Jenkins, TeamCity, Visual Studio Team Services etc. into vREST.
Automatically log issues into bug tracking systems like JIRA, Bugzilla, etc and notify team using Slack.
vREST lets you write test cases to validate your web/mobile application.
Supports Data driven testing. vREST lets you write test cases to validate your web/mobile application.
Schedule your test cases using cronjob utility (in Mac or Linux) or Task Scheduler (in Windows).

#19 WebInject

WebInject is a free tool for automated testing of web applications and web services.
It can be used to test individual system components that have HTTP interfaces (JSP, ASP, CGI, PHP, AJAX, Servlets, HTML Forms, XML/SOAP Web Services, REST, etc).
Offers real-time results display and may also be used for monitoring system response times.
WebInject uses an XML API for defining and loading test cases. You can use WebInject without ever seeing its internal implementation.
Result reports are generated in HTML (for viewing) and XML

#20 Paw

Paw is a full-featured HTTP client that lets you test and describe the APIs you build or consume.
It has a native macOS interface to compose requests, inspect server responses, generate client code and export API definitions.
Easily craft your HTTP requests, try and archive them.
Setup HTTP Headers, URL parameters, form-encoded POST key-values or text.
Get easy-to-read responses with well formatted headers and body.
Sync your projects with your team through Paw for Teams.
Simple request chaining (right-click in the UI, no code or globals).
Great workflow for when writing a fast-changing API

Conclusion

This list can go on and on however we have to stop somewhere. The above list is an effort to explore some different API testing tools which may not be the most popular ones but work well and perform effectively.

Though all API test tools listed above work well and are great options.however, there is no one-size-fit-all tool. There is no perfect tool for all organisations: Every organization has different requirements.

So the key is – carefully considering the current and future project’s requirements and pros and cons of each solution.

API Testing, Key Terminologies and more…..

Rahul — Tue, 31 Mar 2020 23:27:34 +0000

An application programming interface (API) at its core is a formal specification that acts as a guaranteed contract between two separate pieces of software. We can also define API as an interface between two software applications by allowing them to communicate with each other.

APIs have become the focal point of programming improvement, interfacing and transferring information and rationale across frameworks and applications. Fortunately, testing them can enormously improve the productivity of your testing procedure overall, helping you convey programming quicker than at any other time.

So What is API Testing!

Applications often have three layers:

Data layer
Service (API) layer
Presentation (UI) layer

The API layer contains the business logic of an application – the rules of how users can interact with services, data, or functions of the app. Since the API or service layer directly touches both the data layer and the presentation layer, it presents the sweet spot of continuous testing for Testing and Development teams.

API testing involves testing the application programming interfaces (APIs) for functionality, security, Performance. Since APIs lack a GUI, API testing is performed at the Service layer.

Testing types in API Testing

1.Unit Testing: For testing the usefulness of individual activity.

2. UI Testing: For testing the usefulness of UI as a component of start to finish coordination tests to guarantee the UI capacities true to form.

3. Functional Testing: Functional testing is simply a test of specific functions within the code base. These functions represent specific scenarios (successful cases) and error handling.

Due to the nature of the test, we should expect a few stated responses. We should expect either an error (and thus, the appropriate error codes and handling instructions) or a corrected response that bears the material we’ve requested.

Functional testing should be done with the positive scenarios as well as both errata and edge cases.

4. Load Testing: For testing the Functionality and execution under Load conditions.

Load testing takes on a few different scenarios in order to ensure peak performance. The first of these scenarios is called the “baseline“, and tests the API against the theoretical regular traffic the API expects in normal, day-to-day usage.

A second load test is generally done with the theoretical maximum traffic. This is done to ensure that even during full load, methods are in place to safely throttle requests.

Finally, an overload test is performed to test the theoretical maximum and to add 10–20% additional traffic on the top. While this type of testing anticipates some sort of failure, it is more like a test of the API functionality to validate the error code generation and to ensure any failures are handled during high load.

5. Error Detection: For recognising any blunders, for example, special cases and asset spills.

This type of test is focused on the actual running of the API. Other tests are concerned with the result of implementing the API in a scenario, error detection is concerned with the universal results of utilizing the API codebase. These types of tests have one of the following focuses:

• Monitoring: Compiled code is tested for various implementation errors, handler failures to ensure there is no security issue in the code base.

• Execution Errors: The code should respond to valid requests in a predictable, known way, and should fail invalid requests.

• Resource Leaks: Invalid requests, purposefully overflowing commands, and other “illegal but common” types of requests are submitted to the API to test for memory, resource, data, or operational leaks and insecurities.

• Error Detection: The code is put through known failure scenarios to ensure that errors are identified and handled.

6. Usability Testing : It is easy to hit the APIs on different platforms and test the usability of a software.

7. Security Testing: Security testing also includes penetration testing, and fuzz testing.

Security Testing includes validation of encryption methodologies and validating the design of the access control through user rights management and validating authorisation checks.

In Penetration testing, API is attacked by someone with limited working knowledge of the API to assess the threat vector from outside. These attacks can be limited to specific functions, resources, or processes, or can target the entirety of the API.

In Fuzz testing, massive amounts of random data, known as “noise” or “fuzz,” is forced into the system in order to attempt a forced crash, overflow, or any other negative behaviour. This is done to test the API at its absolute limits, and serves somewhat as a “worst case scenario.”

8. Interoperability and WS Compliance testing: This sort of testing applies to SOAP APIs and checks for following two aspects:

Interoperability between SOAP APIs is checked by ensuring conformance to the Web Services Interoperability profiles. By conforming to these guidelines and utilising these tests, interoperability between SOAP APIs can be confirmed

Secondly, WS-* compliance is tested to ensure standards such as WS-Addressing, WS-Discovery, WS-Federation, WS-Policy, WS-Security, and WS-Trust are properly implemented and utilized.

9. Discovery Testing : API testing helps in identifying APIs where the resources are not exposed.

Infiltration Testing: For identifying any vulnerabilities of an application from aggressors.

API Testing Best Practices

1. Deciding What to Test and what not i.e Create test cases for all possible API input combinations to get complete test coverage

2. Use Realistic Data

3. Don’t Repeat Yourself.

4. Write clear tests that easily enable debugging.

5. Design the tests to run under different SUT configuration options.

6. Test Positive and Negative Outcomes

7. Use Data to Drive Dynamic Assertions

8. Track API Responses

9. Test the API for failures; until you get the output as failed.

10. Reuse test cases and monitor the API in production.

A simple API Testing process

Step 1: Create API testing requirements. To be able to plan API tests, we need to answer following questions:

We need to determine testing boundaries and requirements.
Purpose of the API and its target consumer;
Workflow of the application;
Features and functions of the API;
Other APIs this API interacts with;
Aspects of testing;
Problems we’re testing for;
Priorities in testing;
Desired output;
Definition of Pass and Fail.

Step 2: Establish the API test environment. After the functional scope, the next step is setting up an API test environment. It requires the configuration of the servers, databases, and every resource the API interacts with, depending on the project requirements.

Step 3: Make a trial API call. Before starting the testing, perform a Sanity test i.e. make a test API call just to check the API is operational.

Step 4: Define the input parameters. Plan all possible input combinations to verify the results to determine whether the API performs as expected.

Step 5: Create API test cases. After all the preparations are done, it’s the time to write and execute test cases followed by compare actual results with the expected ones. A good practice is to group them by test category. Examples of API test cases may include:

Positive and Negative test values to check the response is Passed or Failed.
When there is no return value in response.
Verification of response after data structure updates;
Validating resources modified by the API call.

Web Service / API Testing Terminologies

1. XML

XML stands for extensible Markup Language
It’s a mark-up language much like HTML
XML was designed to store and transport data
XML was designed to be self-descriptive
It has sender information.
It has receiver information
It has a heading
It has a message body.

2. WSDL

WSDL stands for Web Services Description Language
WSDLs are an XML format that tells you how to access a web service.
WSDL is used to describe web servicesWSDL is written in XML

3. SOAP

SOAP stands for Simple Object Access Protocol. It helps in exchanging structured information between computer networks.
SOAP allows communication between different operating systems using XML.

4. SOA

SOA (Service-oriented Architecture) is a way in which companies can organize software that can be quickly changed to respond to the requirements of the marketplace.
Web Services are small units of software that run in a network. They are typically written to handle a specific business process. Web services can be strung together in multiple ways and used by different applications to create the desired functionality.

5. REST

REST stands for Representational state transfer.
These are web services that provide interoperability between computer systems over the internet.
RESTful Web services provide a predefined set of stateless operations and allow requesting systems to access and manipulate textual representations of web resources.

6. SOAP vs REST

SOAP stands for Simple Object Access Protocol.
SOAP is a protocol. It defines some standards that should be followed strictly.
Web services following SOAP principles are called SOAP Web services
SOAP message request is processed slower as compared to REST.
SOAP supports only XML data format.
SOAP is not very easy to implement so it is preferred less.
SOAP requires more bandwidth and resources.
SOAP Message contains Envelope, body and header.
It does not use web caching mechanism.
SOAP is commonly used in payment gateways, financial and telecommunication services.

REST stands for Representational State Transfer.
REST is an architectural style.
Web services following REST Architectural style are called RESTful Web services
REST message request is processed faster as compared to SOAP.
REST supports data formats like plain text, XML, HTML, JSON, etc.
REST is easier to implement.
REST requires less bandwidth and resources.
REST Message is enclosed in HTTP and uses HTTP Method and URI (Uniform Resource Identifier)
It uses web caching mechanism i.e. data can be cached at the client.
REST is commonly used in social media, web chat and mobile services.

7. Web Services vs APIs

All web services are APIs.
It can only be hosted on IIS.
It is not open source but can be used by any client that understands XML.
It requires a SOAP protocol to receive and send data over the network, so it is not a light-weight architecture.
A Web service uses only three styles of use: SOAP, REST and XML-RPC for communication.
It only supports the HTTP protocol.

All APIs are not web services.
It can be hosted within an application or IIS.
It is open source and it can be used by any client that understands JSON or XML.
It is light-weight architecture and good for devices which have limited bandwidth, like mobile devices.
API may use any style of communication.
It supports the HTTP protocol: URL, Request/Response Headers, caching, versioning, content formats.

8. JSON

JSON stands for JavaScript Object Notation and was designed to be a lightweight data interchange format. JSON is definitely becoming more popular and is now replacing XML in certain situations for API data exchanging. The site www.json.org describes how JSON is built on two structures:
1. “A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed list, or associative array.
2. “An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.”

9. SOAP Message Elements

1) SOAP Envelope – The SOAP Envelope is always the top element in the message.

2) Header – the Header is optional and is the first child element to appear after the envelope. Headers can contain different types of application-specific information like security authentication or session management info.

3) Body – Sometimes referred to as the payload, the Body contains the actual message that shows the information for the recipient of the message.

10. Commonly used HTTP Methods

10.1 GET Method

The HTTP GET method is used to retrieve data. It is a read only call and data cannot be modified. If it is a successful request, without any errors, then a status of code of 200(OK) is received along with the data requested in the particular format which is generally JSON or XML.
GET method is considered a safe method to call as no corruption or modification of data will be done.

10.2 POST Method

The HTTP POST method is a write method. It is mainly used to create a new resource by providing the input in Json or XML format.
When a POST request is made, a new resource is created with the values provided along with it. If it is an error free call, the status returned will be 201.
POST methods are responsible for modification of data and should be carefully handled.
These methods are not idempotent and thus can result in errors or duplicate records if called more than one time.

10.3 PUT Method

The HTTP PUT method is more like an update command. It is used to change the value of any resource whose original value was something else.
PUT methods can also be used to create resources but only if the the request is from the client and not the server.
PUT methods are also not safe as these are responsible for resource modification but if we call the same PUT request again then no changes are made i.e it is idempotent.

10.4 DELETE Method

The HTTP DELETE method, as the name suggests, is used to delete a resource.
If the request is made successfully then the status code returned is 200 (OK). These are also idempotent as if a resource is deleted once it cannot affect it.

11. HTTP Response codes

The HTTP response codes are divided broadly in five categories :
• 1xx – Informational codes
• 2xx – Success codes
• 3xx – Redirect codes
• 4xx – Client error codes
• 5xx – Server error codes

Some of the most common HTTP response codes used with REST are as follows :

200 (OK): This code indicates that the request was made successfully.
201 (Created): This response code indicates that request was successful and a resource was created. It is used to confirm success of a PUT or POST request.
400 (Bad Request) : This code indicates that the data is in incorrect format.
401 (Unauthorized) : This code indicates an authentication error.
405 (Method Not Allowed) : This code indicates that HTTP method used is not supported for this resource.
409 (Conflict) : This code indicates that there is a conflict request to create the same resource twice.
404 (Not Found) :This code indicates that the required resource could not be found.
500 (Internal Server Error) : This code indicates that there is some error on Server side.

The Benefits of API Testing

Early Testing –
With API testing, once the logic is designed, tests can be built to validate the correctness in responses and data. We don’t have to wait for various teams to finish their work or for full applications to be built – test cases are isolated and ready to built immediately.
Easier Test Maintenance –
UIs are constantly changing and moving around based on how they are accessed – browsers, devices, screen orientation, etc. This creates a nightmare scenario where tests are being constantly rewritten to keep up with the actual code in production. API changes are much more controlled and infrequent – often times API definitions files like OpenAPI Spec can help make refactoring tests only a seconds of work.
Faster Time To Resolution –
When API tests fail, we know exactly where our system broke and where the defect can be found. This helps reduce time triaging bugs between builds, integrations, and even different team-members. The small, isolated footprint of an API test is perfect for faster MTTR stats, a valuable KPI for DevOps teams.
Speed and Coverage of Testing –
100 UI tests may take 10 hours to run whereas 100 API tests could be run in 1 hour too. That means we can find and fix more bugs in less time.

Popular API Testing tools

Having the right process, tool and solution for API test are critical for success. Below are the tools which are quite popular in API testing space:

Postman
SOAP UI
REST-Assured
Swagger.io
JMeter
Katalon Studio

I’ll give a list of API Testing tools in next article where we ‘ll see some different API Testing tools.

A Guide to Codeless Automation

Rahul — Fri, 21 Feb 2020 12:56:22 +0000

As we race through the 2020, challenges in front of Testing(more specifically Automation Testing) are increasing by leaps and bounce primarily attributed due to quick Agile development cycle, where development and release cycles involve releasing a set of features and extending those sets in iterated releases. Some of these challenges can be counted as picking the right automation tool, highly Skilled Automation Testing experts, high upfront expanses and testing always being under time constraints.
And it’s very well said that necessity is the mother of invention therefore some of these challenges have given birth to the industry’s newest innovation idea “codeless test automation”.

So what is the Codeless Test Automation?

Codeless testing as the name implies, refers to an approach where test cases are automated with a minimum of code.

Next generation codeless test automation still leverages record and playback, but unlike legacy tools that record “static” test cases, many codeless tools are equipped with machine learning and AI capabilities which provide test re-usability, stability, and minimal test maintenance resulting into the lower training and maintenance upfront cost.

Challenges posed by Traditional Automation Testing tools

A high bar for testing experts – Test automation requires technical skills to design and maintain test automation framework, scripts and resolve technical issues therefore, in order to fulfil the requirements, strong programming skills and solid test automation tools knowledge is required. This is perhaps the most primary challenge in traditional automation tools as it causes multiple other challenges including dependency on specific resources, attrition, high upfront costs.

Frequent releases, Time constraints – For large projects having a fast release cycle, creating a test environment, and writing test scripts take a huge amount of time and resources which ultimately defeats the purpose of automation testing which was originally aimed at reducing testing times and resources.

Multiple setups required – Traditional automation testing involves writing separate execution scripts for each test case, and then executing through a separate setup dedicated to running, monitoring, and logging the test cases.

Significant Investment – As mentioned above, multiple attributes like Training, hiring specific resource, licensing, hardware and software costs make traditional automation tools an expansive affair sometimes and in spite of understanding the ROI in the long run, many people tend to give up test automation looking at the high upfront costs

How the challenges are responded by Codeless Automation?

This problem had led to devise a solution in the form of codeless testing. Codeless testing creates automation tests without writing a single piece of code or minimal code.

This approach has been continuously evolving with new innovation on technology, focusing more on reducing the coding process and making it more user-friendly.

The ultimate aim is to save time of testers in coding the functionalities and focusing more on Business scenarios, functionalities and on the core – bringing the quality through the optimal test coverage.

There are multiple approaches behind codeless testing. The most prominent are:
– Recording tests and playing it thereafter.
– Creating structured test flow diagrams using test blocks provided by tool(s) and later the test flow is used to run tests.

How Codeless Testing Works

Codeless test automation functions quite similar to enterprise codeless platforms. Both take complex tasks traditionally achieved by coding and simplify them through automated code generation.

In the codeless test automation, a tester creates a flow by visually binding elements that represent “clicks” in the application. The tester simply goes through flow of the scenario and the code for the test is generated behind the scenes.

Idea behind the codeless automation is that Testers could focus their efforts on Business, testing functionality and evaluating the user experience rather than struggling with code of the testing framework.

There are multiple approaches to codeless testing. The two most prominent ones involving recording manual tests and playing it back and creating structured test flow diagrams that are used to run tests. The record and play approach is the most popular one right now and especially favoured by web-based products.

Features expected from codeless test automation tools

1. Ability to Build Comprehensive Logic:

Make sure the approach or tool you select allows you to build complex logic and achieve interactions with complex web pages.

Common programming constructs such as conditional blocks, loops, arrays etc., should be easy to handle.

2. Smart element locators

One of the biggest challenges with the code based automation tools like Selenium, UFT, Test Complete is that when the application changes, those automation scripts may fail.

Codeless automation can resolve it by gathering more than one element locator during the recording. In case a locator is no longer identified during playback, the other locators for that element can be explored. Therefore smart element locators are an opportunity for the codeless automation in creating stable tests with less maintenance.

3. Conditional waiting

Record-and-playback tools that require wait times between steps are again a challenge for the testers.

In conditional waiting, approach, scripts wait intelligently until a condition is true and then proceed as soon as possible.

This drastically cuts down on the execution time of the automation suite. Codeless tools should allow testers to insert conditional waiting. Alternatively, the tools took care of this behind the scenes.

4. Flexibility in handling the dynamic applications

The ability to configure the identification criterion for a complex UI element is very important. A good solution would visualise this.

5. Control structures

One of the major limitations of older record-and-playback tools is the absence of control structures, such as loops and conditional clauses.

In coded automation frameworks, testers use loops and if-else clauses to control the flow of scripts. This is a necessary requirement for codeless solutions.

6. Maintain the Business Process Focus

Test representations should reflect business process flow and directly connect with how your end -user exercises the system.

7. Easy assertions

Recording a scenario is fairly easy to do. As the user acts on an application, the tool records those actions.

However, when a user validates the actions, that does not usually automatically include a web action which required a manual action of addition of assertions, again a time-consuming task.

That’s the opportunity for the codeless automation. The process of adding assertions should be as simple as adding the navigational steps.

8. Offer reusability

A codeless approach should incorporate modularity and re-usability at the centre of the design. Oftentimes, this is overlooked. Remember, it is not just about easy test creation, but sustainability across multiple releases and changes that the application goes through.

9. Modification without redo

Some of the existing record-and-playback tools do not allow to edit recorded scenarios simply. In the event of a change in the application flow, you have to re-record the entire scenario.

Some steps are common in almost every application and reused in multiple scenarios e.g. login scenario. In case there is a change in that common step, every test needs to be recorded again which is again a time-consuming task.

Therefore, codeless tools should allow to record common steps that they can then insert into any test flow. This would not only save time, but also promote reusability and make maintenance much simpler.

Codeless automation tools should make the updates in the recorded scenario as simple as they can.

10. Accessibility

The codeless automation system should be accessible, with the information flow managed transparently. It should be simple enough for extended teams to use, as well as contribute.

11. Cross-browser support

Testers should be able to record a scenario once on a given browser, and be able to play that recording on any other major browsers.

12. Reporting

Adequate reporting is a must have requirement for any test automation. An automatically generated report with details on the description, screenshots should be quite helpful for a tester to analyse the failure.

13. Ability to insert code

In spite of providing the seamless records and play feature, there would still be pressing need to customise the code depending on the specific application’s requirements and functionalities. Hence testers should have the ability to insert code within the tests for those edge cases. For extra points, don’t limit this to UI-specific code, such as JavaScript.

14. Extendibility

There are always occasions where out-of-the-box support is not enough. It should be easy to extend the functionality while maintaining the same level of native simplicity.

15. Continuous integration

Older record-and-playback tools required testers to click a button within the tool to execute their tests. In the era of DevOps, test automation is a key ingredient in continuous integration and continuous deployment.

Tests should integrate with such pipelines and automatically execute when triggered. For faster execution times, tests should be capable of running in parallel as well.

Advantages of Codeless Test Automation

Codeless automation testing tools are a perfect solution for the testers to speed up their work. These tools have taken over the market owing to their ease-of-use and speed. Codeless test automation is particularly useful when it comes to agile software development process .

Codeless Test Automation offers several benefits as below.

1. Less training for testers

Needless to say, when testers need to code their tests, they usually undergo specialised training to learn programming. With the introduction of codeless automation, it is easy to train a number of testers in creating automated tests, without any need for scripting skills.

2. Codeless is Economical with more quality

When testers don’t have to spend all their time trying to automate their regression tests by writing code, they have more time for enhancing the test coverage, tests with a variety of data / business data and exploratory testing.

3. Speeds up the testing and overall release pace

When automation is designed with visual UI workflows, all code is generated under the hood making automation easy to learn and implement.
A tester can start test automation as soon as development is completed without wasting time going through complex coding practices and frameworks.
This makes it easier and Compatible with the speed of the development cycle and continuous integration.

4. Lower maintenance and scalable automation

With the Codeless automation, following current business rules and best practices makes automating tests more scalable and maintainable.
For instance, one good example may be introducing the modularity in our test cases. By following this best practice, if changes are made in one of the module, it is not necessary to correct the automation flows every time the system under test changes as codeless test automation depends on native identification of the objects of test use cases.

5. Extend the scope of automation

The codeless automation process makes it easier to support various types of applications, including desktop, web, and virtual applications.
More specifically, this means that automated tests can include various interfaces across applications as part or beyond the project. With this flexibility, it is easy to measure the amount of automation from one to several applications, especially, those that include testing automation between projects

6. Improve adoption with a shared automation solution

Automation Platforms can be used as an one-stop-shop solution for automation needs across an organization. With a UI-based Automation framework, it’s easy to understand flows created by other users.
This comes with the advantage that users can pick up and work on other people’s flows with little or no hand-over required which is not an easy task in the code-based automation framework.

7. Simplified documentation and updates

UI workflows work both as a visual description of a process and as a documentation of how that process is executed and all actions and events are recorded in audit trails.
Generally codeless automation tools come with the audit trails, logs, video as part of the documentation.

8. User-friendly

Code-based automation requires programming skills and hence can often be tricky and complicated to use.
Codeless test automation, on the other hand, has an intuitive GUI, thereby making the process of test creation faster and simpler. Thus, even a manual tester can efficiently perform test automation.

9. Integration Capabilities with a lot of Features

By integrating features such as test management and issue management codeless test automation can transform a testing platform into an entire test suite.

10. Supports Agile

Regardless of their abilities and skills, all testers, developers or even SMEs and project managers can contribute to the testing process using codeless automation.
In summary, all these factors combined means that, with codeless UI automation, there’s no pressing need to understand testing frameworks or the technology underlying an application in order to be able to automate tests.

11. Reusable Test Objects

API First development practices have led to increased usage of service definition files. Codeless provides support to leverage existing API specs while creating and executing test cases.
For web application testing, Page Object model has been used in selenium automation. Codeless allows full page objects to be defined as simple, creating a single source of truth for UI control locators that can be reused across all the test cases.

12. Layered Data Inputs

Defining test data inputs for multiple test scenarios becomes a headache and time-consuming task.

Codeless takes a layered approach to test data definitions, allowing default data values to be embedded within the application.

Individual test steps can then modify the default inputs as required for that specific test case.

At run time, environment and system properties can be invoked to pass execution specific data inputs, such as service host locations, or user credentials.

Disadvantages of Codeless Test Automation

Although codeless automation provides multiple benefits, there are some limitations that are worth listin

1. Maintainability

One can get into a lot of maintainability issues, If the tests are not designed to be modular and reusable, especially when the number of tests drastically grows, and application is continuously changing

2. Training / Documentation

During the analysis of some codeless test automation tools, users have raised the need of more documentation requirement.

3. Features

One prominent feedback for some codeless framework also highlights the lack of few expected features.

4. Debugging

Debugging will be difficult due to the limited facilities from the tool.

Making codeless a reality

Codeless testing, just like code-based continuous testing, relies on four key elements to succeed: Creation, Execution, Lab, and Analysis. Connecting test flows to a reliable creation and skill set matched tool, that can execute the tests reliably at scale from CI or other scheduler against a solid lab, and which then delivers a single pane of glass from a reporting and analysis standpoint, is a significant leap towards mature continuous testing in DevOps.

Maturity – With the tried and tested traditional approach, there are use cases, best practices, and plenty of opportunities for teams to learn from their colleagues who have done it all before.

Trust – To adopt a newer approach like codeless, teams have to make a leap of trust. This is why it’s crucial to seek advice and information from trusted third parties and tech experts in order to make decisions on how best to integrate codeless into your software development processes.

Change of mindset – Same as with any new ways of working, codeless testing calls for a new mindset – and willingness to change which also requires a good leadership, motivation and a comprehensive plan to make it happen.

Co-existence of open test framework tools – Since codeless automation is still getting matured, there are many scenarios, where traditional code-based automation tools will suit more where testing tools like Selenium shouldn’t be overlooked.

Yes, But what are the Codeless Automation tools?

Here is the list of 10 useful codeless automation tools which might be helpful to understand the presence of codeless trend.

Conclusion

First of all, when we say Codeless, it’s not codeless. It’s hiding code under the hood. The instructions given by the testers on the GUI level are translated into actual code by the solution and then executed.

There are plenty advantages of codeless automation tools mainly the time and cost saving whereas few disadvantages include need of more modularity, features and documentation.

In conclusion, codeless automation is the future. Though the benefits of codeless outweigh the issues with them, however since codeless automation is still evolving, coded automation tools still have their own importance, need and benefits.

Before we leave, here are 2 good resources for the Job interviews and resumes/cover letter preparation:

40 Most Useful Chrome extensions for Website Testing

Rahul — Tue, 28 Jan 2020 12:06:35 +0000

When it comes to testing especially Manual / Functional testing people think easy and quick. Ok but what about the value addition? This value addition can be made even in smaller contributions with the sole purpose of Less errors, less defect whereas high user-friendliness and higher quality resulting into increased Business and reputation.

One of the Business value can be added by using some tools or add ins on our favourite browser – Google Chrome which we may or may not be aware of. These extensions are quick to install, easy to use however outcome may positively astonish you.

So, below are 40 such Chrome extensions which we can use in order to enhance the overall testing quality of our / client’s website.

1. WhatFont

As testers, we must verify cosmetic issues like which font family and font size is used for a particular section on any web page.
WhatFont is a simple extension which sticks to its name – easily find fonts on any web page. Simply hover on any font you want to identify on a web page, and it shows which font is used. If you need further information about which service is used to serve the web fonts, WhatFont also provides this information.
With this extension, you could inspect web fonts by just hovering on them. It is that simple and elegant.
It also detects the services used for serving the web fonts. Supports Typekit and Google Font API.
Cosmetic issues like font size, style, and color are also to be verified by the software testers. WhatFont enables the software testers to easily identify a particular text on a web page. All you have to do is select the font about which you want to gather information.

2. ColorZilla

It is similar to WhatFont, but the only difference is that it identifies colors instead of texts and fonts.
The extension comes in handy when you want to match the color scheme of a web page to meet the design specifications. ColorZilla is a color picker chrome extension that allows you to find the exact color used on any web page right within Chrome browser.
ColorZilla is really useful when you needs to test whether the color used on a web page matches design specifications.
ColorZilla includes a Color Picker, Eye Dropper, Gradient Generator and many additional advanced color tools.
With ColorZilla you can get a color reading from any point in your browser, quickly adjust this color and paste it into another program. You can analyze the page and inspect a palette of its colors. You can create advanced multi-stop CSS gradients.

3. Spell Checker

Spell Checker Chrome extension does exactly what it says on the skin! It checks spellings for all words on a web page and suggests corrections for misspelled words.
It is a simple chrome extension for copy-verification during web testing and supports 12 languages.
Highlights misspelled words on the active webpage.
Quickly spell check webpages.
Supports English dictionary.
Marks misspelled words inline on the page with a red underline.
Safe – plugin requires no special permissions or external requests.

4. IE Tab

IE here stands for internet explorer. This Chrome extension allows you to test the compatibility of your web page with all versions of Internet Explorer. All this is done on the Chrome browser.
While many people have moved on from Internet Explorer (IE), a surprising number still use the old-school browser. It gets around 3% market share – more even than Firefox.
You don’t have to go back and download IE to do that, the IE Tab extension takes care of it for you. You can run IE entirely within Chrome and see precisely how your website looks and works in the legacy browser.
IE Tab exactly emulates IE by using the IE rendering engine directly within Chrome. This will enable you to use ActiveX controls and test your web pages with different versions of IE (IE6, IE7, IE8, or IE9).

5. Session Manager

Software testers are required to open the same website links and URL’s multiple times in a day. With Session Manager, the tester can create a group of websites that they open together every so often.
Session Manager allows the tester to open and close that group of websites with one click. Also, you can save, delete, restore, and update the group.
Session Manager is a simple yet powerful extension that makes it quick and easy to save, update, remove, and restore sets of tabs.

6. Check My Links

Broken links are a normal part of life, but that doesn’t mean you should be OK with them becoming a normal part of your website. Even for small websites, going through your site page by page to check all the links is a large and tedious task. For larger ones, it’s downright herculean.
Check My Links is a link checker that crawls through your webpage and looks for broken links.
Check My Links automates the process and saves you the trouble of having to do it manually. Each time you run it, it checks the full site and spits out a report showing all the broken links you need to update.
‘Check My Links’ is an extension developed primarily for web designers, developers and content editors.

7. Web Developer Form Filler

Testing out forms can be a tedious part of QA testing your website, but forms are also one of the most important parts of your website.
They’re used to collect information on leads, process purchases, and enable trackable contact with your customers. Form testing is not a step you can skip.
Web Developer Form Filler is a website QA testing tool that’s as straightforward to use as its name is at describing what it does. Fill in the values you want to use to test a form, and the extension will automatically fill them in for you each time you go to test. You can set up multiple sets of values based on how many different forms you have to fill out. It can significantly increase the speed of form testing across your website.

8. Lightshot

Lightshot is a light and handy screen capture tool that allows you to take screenshots for test evidence and customize them by adding annotations.
It has a simple interface and is light-weight. You can save the screenshots to your local machine or upload them to the cloud (which you can share as a link with others).

9. Awesome Screenshot

Most of the software testers make use of this Chrome extension to grab screenshots of the test as evidence. Testers can also grab the screenshot of a region on the screen, annotate, highlight and crop it. The screenshot can be easily shared with other testers or you can just save it to your desktop.
Screen Capture full page screenshot. Screen recorder for screencast. Annotate and share to Trello/slack etc.
Capture all or part of any web page. Add annotations, comments, blur sensitive info, and share with one-click uploads. Now with free desktop capture!

10. Screencastify

Another extension which should be a part of a tester’s toolkit. Screencastify is a screen-recorder extension for chrome which you can use while testing.
It allows capturing user behavior on any web page as a video which you can share with developers as test evidence. Just press record and the content of your tab is recorded.
Record
– Desktop, browser tab or webcam capture
– Narrate with your microphone’s audio
– Embed your webcam into the recording
Edit
– Trim the start and end of your videos
– Draw on your tab with the pen tool
– Merge, crop, and add text to videos (upgrade required)
Share
– Videos autosave to Google Drive
– Save and manage videos locally
– Export as MP4, animated GIF, or MP3

11. Edit This Cookie

EditThisCookie is a cookie manager. You can add, delete, edit, search, protect and block cookies!
If you are testing A/B versions of a web application or want to check if a feature is accessible to a defined segment of users, this is one chrome extension which will save you a lot of time.
Edit This Cookie provides a sleek, well-organized interface which allows you to manage your browser cookies like a pro. It shows set cookies and their values – you can change any of the current set values or add, delete, search, protect or block cookies. Using this extension saves you the hassle of going through the Chrome settings screen to search for cookies for a particular site.

12. Cookie Editor

Cookie Editor is another chrome extension which you can use to edit cookies during testing. Like it says on the tin, it allows you to edit cookies – you can add, delete, edit, protect, block and search cookies. It also enables you to export cookies in JSON format.
You can easily create, edit and delete a cookie for the current page that you are visiting.
There is also a handy button to mass delete all the cookies for the current page.
You can also Import and/or Export your cookies in a text format for easy sharing or backup.

13. Clear Cache

Find it a hassle to clear browser cache in chrome browser when you in the middle of testing? This chrome extension allows you to clear browser cache from the toolbar avoiding the longer route.
You can customize it to control which data you want to clear – app cache, downloads, file systems, form data, browsing history, local storage, passwords etc.
You can customize what and how much of your data you want to clear on the options page, including: App Cache, Cache, Cookies, Downloads, File Systems, Form Data, History, Indexed DB, Local Storage, Plugin Data, Passwords and WebSQL.

14. Cache Killer

Cache Killer is a useful chrome plugin for software testers that allows you to easily disable caching in Chrome. Unlike Clear Cache, when enabled it clears browser cache before loading a page so that you always get the latest version of the code from the server.
If you are testing caching feature for your application, you can disable this add-on with a single click.
Clears the browser cache automatically. On or Off extension by single click
This clears the browser cache automatically on opening a new tab or refreshing a tab. You can enable or disable the extension on single click.

15. Screen Ruler

Screen ruler is an essential extension which allows for you to measure the height and width of an object on the screen.
Using Screen Ruler, software testers can find out defects in the UI. Also, this helps to create a web application with an appropriate number of pixels.
Using Screen Ruler chrome extension while testing, you can measure object height, width, padding on either side by simply placing it over the element and dragging to draw out a ruler in the desired direction.
Screen Ruler is really useful chrome extension for Web UI testing as it helps you log defects which otherwise wouldn’t be visible and helps make sure a pixel-perfect web application.

16. Perfect Pixel

If you want that the design on the web page is a replica of some other page, this chrome extension will get the job done for you. Pixel comparison with other pages is possible with this Chrome extension.
PerfectPixel allows developers to put a semi-transparent image overlay over the top of the developed HTML and perform pixel perfect comparison between them.

17. Resolution Test

Resolution Test is an alternate option for QA testing out how your website looks in different screen sizes. Using it is simple. Once you’ve installed it, click on the plugin image in the top right browser menu.
You’ll get a dropdown menu with a list of different screen resolutions, and you can click on each to see how your website looks in it.
Resolution Test changes the size of the browser window for developers to preview their websites in different screen resolutions. It includes a list of commonly used resolutions and the ability to customise that list.

18. Window Resizer

Testing out your website on just your own computer or device is an easy mistake for website owners to make. But the people who come to visit your website will be coming from an array of devices with different screen sizes.
The Window Resizer plugin is an easy way to test out how your website looks in different window sizes. The tool comes with some preset testing options, based on the most common sizes for desktop and mobile devices.
If important parts of a page drop below the fold or your website looks funny in different sizes, the extension will help you spot it and update your design to fix the issue.
The resolutions list is completely customizable (add/delete/re-order).
You can set the window’s width/height, window’s position, preset icon (phone, tablet, laptop, desktop) and even the option to apply the new dimensions to the entire window or just to the viewport.

19. Exploratory Testing Chrome Extension

Another simple chrome extension for exploratory testing that makes web exploratory testing easier. It allows you to add annotations in the form of bugs, ideas, notes and questions which you can see in a report along with screenshots for the session.
A Chrome extension designed for making web exploratory testing easier. Features include:
Report bugs, ideas, notes and questions easily
Take screenshots during the session. Keep focused
URL will be tracked automatically
See session results in a report
Save and import session
Export session to JSON,CSV or HTML

20. Form Filler

This chrome extension makes filling form fields a child’s play and proves useful while exploratory testing. You can set up hot-keys to fill web forms using a single key.
Fill all inputs in a page with dummy data.
The purpose of this extension is to help developers and testers test their forms quickly and easily.
This extension allows you to fill all form inputs (textboxes, textareas, radio buttons, dropdowns, etc.) with random/dummy data.
This extension is a must for developers and testers who work with forms as it avoids the need for manually entering values in fields.

21. QMetry Exploratory Testing

Test while you Explore. Save efforts with Automated documentation. Report Bugs with screen shots and reduce streamline communication
This extension works with QMetry Test Management standalone as well as JIRA app.
Exploratory Testing with Automated Documentation
As you explore, extension automatically records your action, captures screen shots and generated automated Test Case documentation.
Detailed Bug Reporting
Found a bug while exploring? Export all bug details with screenshot and environment details with single click.
Save Communication Time & collaborate better

22. Postman

You all might be aware of Postman tool in API space.
Postman Chrome Extension is the go-to tool for testing API requests – particularly RESTful APIs.
It gives you the ability to set-up and switch environment variables on the fly saves plenty of time.
You can group test API requests into collections which sync to the cloud making them accessible to others for team collaboration. Postman has a low learning curve too which means you will be testing APIs in no time.

23. Advanced RESTClient

Advanced RESTClient is another chrome extension for API testing. It is not as feature-rich as Postman but is great for a start into API Testing.
REST Client allows you create and test custom API requests with a simple interface.
The only REST client that makes connection directly on socket giving you full control over the connection and request/response headers.

24. Resteasy

Resteasy is another option for testing RESTful APIs within Chrome. It offers similar functionality to Advanced REST Client chrome extension. So, you can try both and choose your favourite.
RestEasy is a tool that is targeted against developers who perform RESTful API calls against third party services.
RestEasy is a tool that is targeted against developers who perform RESTful API calls against third party services. It allows developers a way of being able to debug and analyze an Http request/response. RestEasy is built using Google Chrome’s Extension API framework and includes common technologies such as Html and JavaScript.

25. JSONView

If you are testing RESTful APIs, reading raw JSON data can be daunting.
JSONView chrome extension allows you to view formatted JSON in tree view within Chrome browser making it easier to read and verify.

Cyber security isometric flowchart with hardware protection symbols on blue background isometric vector illustration

26. XSS Rays

XSS Rays is a useful chrome extension used widely by penetration testers for security testing. It is a pure Javascript XSS scanner which helps in identifying XSS vulnerabilities in any website. XSS Rays parses all the
XSS Rays is a pure Javascript Cross-Site Scripting (XSS) scanner, originally developed by Gareth Heyes in 2009.
XSS Rays parses all the links and forms of the page where it has been loaded and checks for XSS on the GET, POST parameters, and the URI path by creating hidden iFrames.

27. Request Maker

Request Maker is a core penetration testing chrome extension using which you can create new requests, capture requests made by web pages, tamper with the URLs, modify headers and POST data.
The feature I like the most in Request Make is – you can bookmark the requests on the click of a button which is a great time-saver.
Request Maker doesn’t fill the log with useless information about images and style sheets.
The logs are tab-specific, displayed in the page action popup, and the requests are bookmarkable. After sending a request you can just click on the bookmark button like you would on any other page.

28. d3coder

d3coder is another chrome extension for penetration testing.
You can encode and decode selected text via context menu added within Chrome itself saving you plenty of time. Copy text to your clipboard and select a conversion from the menu.
After the conversion, d3coder copies the new text into your clipboard. d3coder chrome extension is a great plugin for penetration testing allowing various types of encoding/decoding like base64, rot13, CRC32 hashing, UNIX timestamp conversion.

29. Site Spider

Site spider is site crawler chrome extension which reports broken link on any web page.
This extension saves a lot of time when testing software as you don’t have to manually check if all links are working.
It also gives you the ability to restrict the spider by adding restrictions and regular expressions.
Site Spider is open source which means you can change it as per your needs.

30. WAVE Evaluation Tool

WAVE is a great web accessibility evaluation tool which is used to test any website against WCAG guidelines.
WAVE Evaluation tool is a great chrome browser extension for accessibility testing as it adds WAVE capabilities within your browser and provides visual feedback about the accessibility of a web page by injecting icons and indicators into the web page.

31. Accessibility Developer Tools

WAVE is a great web accessibility evaluation tool which is used to test any website against WCAG guidelines.
WAVE Evaluation tool is a great chrome browser extension for accessibility testing as it adds WAVE capabilities within your browser and provides visual feedback about the accessibility of a web page by injecting icons and indicators into the web page.

32. aXe

aXe is another chrome extension that adds automated accessibility testing capabilities to Chrome browser. It is a lightweight extension based on aXe javascript library.
aXe reports errors for any deviations from WCAG 2.0 (W3C Web Content Accessibility Guidelines) and Section 508 (US Federal Procurement Standard).
The extension itself is very light and full of handy features. If aXe supervises the codes that you write on the basis of Web Content Accessibility Guidelines and communicates and mistakes or deviations on the same instant.

33. ARIA Validator

ARIA Validator adds a button to Chrome that validates the ARIA implementation on any webpage. For each frame it can read on the page, it will open a new tab showing errors and warnings.
Adds a button to Chrome that you can click when you want to validate the ARIA implementation of the HTML page you are viewing. Also checks for duplicate IDs in your HTML.
It will open a new tab showing errors and warnings for each frame it can read on the page.

34. Total Validator

The range of validations includes: HTML and XHTML (28 different standards), CSS (1-3), Accessibility (WCAG 1, WCAG2, US-508), a broken links checker, and a multi-language spell checker.
Allows you to validate web pages or entire sites as you surf along.
The range of validations includes: HTML and XHTML (28 different standards), CSS (1-3), Accessibility (WCAG 1, WCAG2, US-508), a broken links checker, and a multi-language spell checker.
It works in combination with a copy of the Total Validator application, which must be already pre-installed.

35. PACT Engine

PACT Engine fetches the violations of WCAG 2.0 and Section 508. Pact Engine will also axelerate the best practices of the web development which will help developer to maintain code standards.
ACT will differentiate accessibility issues Level wise and WCAG guidelines wise, which will help both development team and client as well.
PACT is lightweight chrome extension built up on core javascript, hence pact will show accessibility issues in a blink.
PACT differentiates itself from other approaches and rules repositories in the following ways:
PACT is developed as browser plugin – Any web page or application can be tested.
PACT will differentiate – Level A, AA, AAA , Section 508 & WCAG success criteria.
PACT will export all violations to either EXCEL or JIRA – easy to track all the issues.
Dynamically violations will get updated based on dynamic content.

36. Spectrum

Instantly test your web page with different types of color vision deficiency.
Color Vision Deﬁciency (CVD) affects people’s ability to distinguish certain colors. Estimates indicate that approximately 200 million people worldwide are affected by some kind of CVD.
This extension helps you to test web pages for people with different types of CVD. It’s particularly useful for websites with data visualizations, because some colors may not be distinguishable from other colors in the charts.

37. Tennon Check

Web accessibility testing has a problem – testing after the fact which Tenon seeks to solve.
Tenon is a one of a kind accessibility testing tool in that it is aimed at offering unprecedented flexibility in tooling for designers, developers, testers and content authors.
Tenon achieves these goals via its API which can be seamlessly integrated into existing toolset.
Tenon can be used on any type of IDE, CMS, automated build and deploy tool you use, unit testing, acceptance testing, or issue tracking tool.

38. Siteimprove Accessibility Checker

The Siteimprove Accessibility Checker is the tool to evaluate any web page for accessibility issues at any given time.

It provides intuitive, visual feedback about content by highlighting detected issues right on the page and gives you:
An immediate overview of your page’s accessibility issues
A clear explanations of how they affect your users
Specific recommendations on how to fix them
All analysis is done entirely within the Chrome browser, allowing secure evaluation of password-protected or non-public pages, multi-step forms, and dynamic content.

39. Performance-Analyser

When site performance is one of the most important aspects of keeping customers on your website, ensuring fast page speeds is a priority.
Get insight into the performance of the website in the current tab – sort of a mini live version of WebPageTest.
Performance-Analyser (aka Performance-Analyzer) helps to analyze the current page through the Resource, Navigation and User Timing APIs – see requests by type, domain, load times, marks and more – instantly.

40. BlazeMeter

Blazemeter is a very popular tool in Performance area. You don’t have to be a performance tester to use Blaze meter extension. Blaze Meter Chrome extension enables you to – Record. Browse. Upload. Run.
The Chrome extension records all of the HTTP/S requests and actual user interactions that your browser sends, creates a synced JMeter and Selenium scripts, and automatically uploads it to BlazeMeter, where you can execute it with a single click.
No need to install JMeter nor Selenium to record or run your performance test, the Chrome extension will create the JMeter and Selenium scripts automatically.
The extension will also allow you to debug and test your recorded Selenium session on your local Chrome browser.
By using JMeter locally, you can debug and enhance the JMX file create by the Chrome Extension to insure your script is ready to go.

Afterthought

These 40 Website testing tools are just a reference though a significant list which you can use as a launchpad. You must expand it through your exploration and specific requirements and keep adding value to the organization and clients you work for and the Testing quality overall.

Salesforce Testing Tutorial: A Beginners Guide

Rahul — Thu, 02 Jan 2020 22:10:35 +0000

Salesforce, as we all are aware, offer best-integrated development Environments, and programming tools.

After the 2 key grand acquisitions of Tablue and Mulesoft, company is expecting its revenue to be double in size by fiscal 2024, which translates to revenue between $34 to $35 billion.

Well, how about the Testing of Salesforce applications? What types of Testing we can perform, what challenges are there and the testing best practices specific to Salesforce apps? Made an effort to give a quick overview of Salesforce Testing. Hope it would add some value to the Salesforce beginners.

What is Salesforce!

Before starting this Salesforce testing tutorial, lets see what’s Salesforce! Salesforce.com, Inc. (usually abbreviated as SF or SFDC for Salesforce.com) is an American cloud-based software company headquartered in San Francisco, California.
Salesforce was ranked first in Fortune’s 100 Best Companies to Work For in 2018 and second in 2019.
The primary purpose of a CRM tool is to sustain the relationship of an organization with its customers once the product is delivered to customers.
Over the time, along with providing CRM services, Salesforce started to offer cloud storage as well, which reduced the hassle of maintaining physical servers for data storage of web applications.

What is SalesForce Testing?

Salesforce testing is a validation of the configuration and customisation performed in Salesforce.
Salesforce CRM is built on a platform development language called APEX.
It also provides built-in unit test case for developers to validate their code. Manual Testing in Salesforce is performed by the QA team which includes happy flow testing, functional testing, integration testing, regression testing and system testing.
Automation Testing in Salesforce can be done by any of these tools available in the marketâ€”Provar, AutoRABIT, Assure Click, Selenium and QTP. Selenium is the best choice as it is the open source tool.
Functional flows report based on status of test cases, where testers are required to create the functional flows to understand the functionality of application.
Process builders to check the behaviour of the system, by giving different entry and rule criteria.
Workflows to check the functionality of time-based events.

Why Salesforce testing?

More than 82,000 companies use Salesforce platform worldwide.
Helps to maintain a positive relationship with the customers.
Enhanced communication between customers and organisations.
Automation of daily tasks.
The productivity of the developers would be increased as Salesforce provides inbuilt objects to reduce the development effort.
No additional software is required to use Salesforce.
Developers can reuse the existing applications through the built-in Salesforce app store named App Exchange. Salesforce also allows the developers to build their own custom applications.
Inbuilt reporting mechanism.
Salesforce administrator can create internal users within the Salesforce platform.
Allows you to check that configuration and code is functional.
Helps you to confirm that the system’s initial build meets the agreed requirement
Process builders feature helps you to check the working condition and behaviour of the system
Workflows allow you to check the functionality of time-based events

Salesforce Key Terminology

Salesforce Objects
Salesforce Objects are database tables which permit us to store data specific to the organization. Standard Objects: Standard objects are that
Salesforce Standard Objects
Salesforce comes with a large number of standard objects that are integral to Salesforce, but you can also create custom objects.
Salesforce Custom Objects
All the items created by external forces and not by Salesforce.com are known as custom objects
Relationships
Master-Detail, Lookup
Record
This is made up of a bunch of fields that hold information to describe a specific item. For example, a contact record typically contains fields pertinent to a person, including name, title, phone number, and e-mail address. A record is displayed on a detail page
Accounts
Companies or individuals that are involved in a business relationship. These can be partners, customers or competitors.
Contacts
Individuals within the accounts.
Opportunity
An event or activity for revenue generation.
Case
A problem which a customer may have raised.
Solution
The description of issues and their resolution.
Forecast
Estimated quarterly revenue of the organization.
Folders
Holds the documents and determines the access to all the documents within it.
Report
Analysis of all data that is stored in custom or standard objects.
Dashboard
Graphical data or groups of charts, generated from reports.
Activity
It includes calendar events and tasks.
Products
These refers to items that are sold to customers.
Campaign
Marketing projects.
Lead
The companies that are interested in your product.

Levels of Testing in Salesforce

Unit Testing

Unit testing process is conducted by Apex developers. It involves writing clauses in their code which automatically tests its coverage.
It helps you to evaluates how many records of data are effected, so that the code would successfully run on in that environment.
To deploy Apex code into a Production environment your code coverage ratio should be minimum 78%.

System Testing

Performed by an expert team of Salesforce consultants.
Involves testing of the developed systemâ€s technical processes from start to finish.
Involves test script based on specific outputs.
Allows you to troubleshoot a problem with automated rules in the system e,g, workflow, validation, assignment.

UAT Testing

It Is conducted by the end users.
Performed with test scripts which mimic the real case business scenarios with expected business outcome.
The desired output should be that the client confirms that the system is fit for purpose

Production Verification Testing

It is a repeat of system testing in the Production environment, performed by Salesforce Consultants and end users.
Production testing is done to test whether config and code have been correctly deployed from sandbox to production environment.
The end users should perform the steps from UAT testing again post deployment.

Regression Testing

Performed by System users, the main objective is to verify that the released code and configuration features do not affect the existing processes in the system.
It is done after the release of features that are intended for specific set of users and tested by system users to whom the release is not intended.
The main purpose of the testing is to confirm that the system users for whom the released features are not intended can perform their day to day activities without any issue.

Salesforce Testing Process

The testing process for Salesforce could be the same as of testing any web application.

But, going a step ahead, a tester should have a clear knowledge of:

Sales and Service clouds
Standard and Custom Objects
Salesforce Security concepts
CPQ Configure, Price, Quote
Workflows and Process Builder
Reports and Dashboards
Lightening module

Once the code is deployed in a Sandbox environment and approved to be ready for release, the code would be moved into production from the Sandbox environment.

It is assumed that the tester has the basic knowledge of all the terms used in Salesforce before proceeding with testing.

Salesforce Test Automation Tools

Selenium web driver
HP Unified Functional Testing (UFT)
Cucumber
Force.com IDE (Eclipse-based)
Change Sets (Cloud Deploy)
Ant/force.com migration tool
Provar

Salesforce Load Testing

Salesforce load testing Challenges

The multi-tenant environment of software as a service platforms.
Scheduling complexities.
System governors, which throttle performance any time running code fails to meet varied criteria established by Salesforce.
A total prohibition by Salesforce against stress testing.

And Potential Solutions

Benchmark the environment to establish a baseline for performance in all load scenarios.
Evaluate the current test plan and adjust it if required to ensure it adheres to best practices.
Confirm resources are enough to achieve the desired result and adjust them if not. This includes not only appropriate server resources to handle the extreme loads that may be generated, but also personnel resources.
Aggregate the necessary information to meet Test Plan requirements imposed by Salesforce and submit it in the format and timeframe required by Salesforce.com Customer Service.
Monitor tests to ensure performance and confirm in-range results. Remediate problems quickly to avoid failureâ€”or at the minimum, test plan shutdown by Salesforce.The Final Outcome
Implementing and managing a quality-focused performance testing program for Salesforce customizations isnâ€t easy, but it is essential. Consider this:
Studies by reputable organizations (such as the Pew Research Center) show worker productivity rises measurably when technology tools, such as software, work as expected.
Poorly functioning or failed customizations, especially for Salesforce, will almost certainly have a negative impact on sales and customer service.

Salesforce Security Testing : Best Practices

Turn on IP restriction for user logins to minimise the risk of unauthorised access in case of compromised accounts
Turn on multi-factor authentication for all users to further reduce the risk of unauthorised access
Make organization-wide sharing rules as restrictive as possible while allowing normal business functions and use role hierarchies, sharing rules, permission sets, etc.
Require secure passwords that combine uppercase letters, lowercase letters, numbers, and symbols, and require a minimum of 8 characters.
Set a maximum incorrect login attempt to between 3 and 5 times.
Enable obscured secret answers for password resets.
Force re-login upon session timeout but enable session time out warning popup.
Keep the session timeout time frame as low as possible without annoying your Salesforce user base.
Disable caching and autocomplete on login page.
Expire user passwords within 90 days of creating it.
Enforce password history so same password isnâ€t used until at least 5 new passwords have been used since the last time the given password was used.
Passwords should not contain the word â€˜passwordâ€.
If using platform encryption, regularly generate a new tenant secret, which will generate a new encryption key.
When destroying encryption keys, make sure all data encrypted with that key is decrypted first.
Re-encrypt already encrypted data with the latest key if theyâ€re using old keys, even if the old key is archived and not destroyed.

Salesforce Exploratory Testing

Like exploratory testing on other web applications, Salesforce Testing should also involve:

Validating the consistency of data across multiple screens.
Negative test flows, such as deleting the default data generated.
User input validation on the form fields.
Cross browser compatibility testing to ensure if the rendering of data is correct across multiple browsers.
Maximum length validation for each of the editable input fields along with the invalid data validation.
Error message validation when invalid data is passed onto the applications.
Amount field validation using Boundary Value Analysis and Equivalence test techniques.
Reports and dashboard testing to test various test data parameters.
API testing for integrated third-party applications.

Roles and Responsibilities of a Salesforce Tester

The tester needs to make sure that the code coverage does not go below 75% as per the standard Salesforce rule.
Conduct Smoke Testing to make sure all the major functionalities are functioning as per requirement.
Create both positive and negative Test Scenarios.
Conduct role-based testing to ensure the consistency of data with various user roles.
Able to perform equivalence Partitioning and Boundary Value Analysis.
Perform compatibility testing of the third-party applications integrated with Salesforce if any.
Testers are also needed to work on the application and gain the understanding of its functionality to create functional map.
A tester needs to be familiar with load testing tools such as J Meter to validate the complex flows that produce inconsistent results in Salesforce.
The tester needs to execute role-based test cases to ensure the consistency of data.
Performs compatibility test in case Salesforce integrated with third-party apps.
Clear communication with the development team and Business, to understand the customisation Salesforce features.
Knowledge of Apex.

Salesforce Testing challenges

It’s not an easy task to test advanced features like Visual force, Salesforce or Service Cloud Console.
Writing test cases with different roles and mentioning the settings
You need to rewrite and rerun your classic tests for the Lightning UI.
Some of the standard functionalities can’t be removed even they are not in use.
GUI tests don’t work when we switch to the test environment
Automated tests should able to work in all your test environments
Field Locators are how you automate tests, find the field or button on a page. There is an issue creating field locators for the Salesforce screens as some field IDs differ between Organisations.
Testing of Visual force pages through automation. The issue lies in creating field locators reliably on a page. Salesforce will generate the Ids at run time which means any change to our APEX code, leads to the change of field locators based on Ids which need constant maintenance.
Last but not the least, the most important thing is to understand the Salesforce administration without fail.

Best practice for Salesforce testing

Collate information for the testing
A comprehensive list of a prioritised test case scenarios document must be prepared that covers all possible business case scenarios and technical exceptions. The document should also include the expected result of each scenario and action. Accordingly the test data has to be created for testing the defined scenarios.
Executing the tests
The testing document scenarios must be executed, and the results have to be documented for testing audit. Any failure scenario or action in the test case must be escalated to the responsible team and retested after rectification. While testing code it is recommended that the testing be done in isolation to ensure that the code interacts only with the relevant data for the tests.
Use valid user profile
The user level configurations and permission sets associated with the profiles commands which systems and what level of visibility the user is entitled to. So it is highly recommended that the testing be done with respective valid user profiles and check for consistency with the business processes.
Test both Negative and Positive paths
The system testing and UAT testing should be done for both positive and negative flows of every test case scenario. Every test result should comply with recorded expected outcome for the same with zero deviation.
Managing the tests
All the different tests and scenarios should ensure that the built application or feature is in sync with the existing manual or systemic operations. So it is mandatory to check the results against reference data that directs the quality of the final product. The tests executed should refer to this data and final results should conform to the same.

Afterthoughtsâ€¦…

Whatâ€s in Salesforce for Testers

Salesforce has a dedicated community of developers and testers whom one can connect to in the hour of need.
Testers doing Salesforce testing are often called â€˜Quality Engineersâ€ compared to the â€˜Quality Assuranceâ€ tester in general
Salesforce can be used using a web browser without the need to install additional hardware or software, which makes it the ideal choice for organisations worldwide. Career growth aspects for Salesforce platform testers are very rapid.
With more than 90K companies using Salesforce platform, testers can opt for SalesForce.com administration certification or SalesForce.com developer certification and improvise their career.

Further resources to gain more details about Salesforce, click on the below links:

Big Data Testing Tutorial | A Tester Guide…..

Rahul — Sat, 07 Dec 2019 11:45:03 +0000

So First Thing First - What is big data?

In the era of data-driven decision-making and the proliferation of big data, the need for reliable and accurate data has become paramount. Organizations today rely on vast amounts of data to gain insights, make informed decisions, and maintain a competitive edge. This has given rise to the critical field of Big Data Testing, a specialized form of quality assurance that ensures data correctness, integrity, and performance within large datasets. In this comprehensive tutorial, we will delve into the world of Big Data Testing, exploring its significance, challenges, best practices, and tools.

Gartner defines Big Data as, “High-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision making, and process automation”.

In simple words, big data means a large volume of data. For instance, Facebook generates 4 Petabytes of data every day with 1.9 billion active users and millions of comments, images and videos updated or viewed every second.

The huge collected big data can be of three formats, as:

Structured
Unstructured
Semi-Structured

Structured

Unstructured

Semi-structured

Highly organised data that can be retrieved using simple
queries.
Examples are – Database, Data warehouse ER and CRM. Data types of
predefined structure come under this category.

It does not have any predefined format and it is difficult to
store and retrieve such type of data.
For example images, videos, word
documents, presentations, mp3 files and sensor data.

The type of data which are no rigidly organised and contain tags
and Metadata come under these types.
Examples: XML, CSV and JavaScript Object Notation (JSON).

So What is Big Data Testing?

Big Data Testing is a specialized process of verifying and validating the data in massive datasets, often characterized by volume, velocity, variety, and complexity.
It ensures the quality, accuracy, and consistency of data while dealing with diverse data sources, formats, and structures.
The primary objectives of Big Data Testing include:

1. Data Accuracy: Confirming that data values are correct and consistent.
2. Data Completeness: Verifying that all the expected data is present in the dataset.
3. Data Consistency: Ensuring data consistency across various data sources and timeframes.
4. Data Integrity: Detecting data anomalies, errors, and inconsistencies.
5. Data Performance: Assessing data processing speed and efficiency.

The primary characteristics of Big Data are:

Volume: Volume denotes the size of the data.
Velocity: Velocity is speed at which data is generated.
Variety: Variety specifies the types of data generated.
Veracity: Veracity tells how trustworthy the data is.
Value: Value gives us the ideas of how big data can be turned into a useful entity in business.

The Need for Big Data Testing

In the fast-paced digital age, data has become the lifeblood of businesses and organizations. The ability to harness and leverage massive volumes of data is a competitive advantage that fuels innovation, enhances decision-making, and optimizes operations. However, with great data comes great responsibility – the responsibility to ensure data quality, reliability, and accuracy. This is where Big Data Testing plays a pivotal role. In this article, we will explore why there is an imperative need for Big Data Testing in today’s data-driven world.

Data-Driven Decision-Making:

In an era where data drives critical business decisions, the integrity and quality of data are non-negotiable. Decisions related to product development, marketing strategies, customer service, and resource allocation are increasingly data-driven. Flawed or inaccurate data can lead to misguided decisions, resulting in missed opportunities, financial losses, and even reputational damage.

Big Data Testing is crucial to ensure that the data underpinning these decisions is accurate, consistent, and reliable. Without rigorous testing, organizations are essentially navigating a treacherous landscape blindfolded, with the potential for costly missteps.

Trust in Data:

Trust is a cornerstone of any data-driven organization. When executives, analysts, and data scientists cannot trust the data they work with, the foundation of decision-making crumbles. Inaccurate data can erode trust in analytics and dashboards, leading to skepticism and inefficiencies within the organization.

Big Data Testing helps maintain data trust by identifying and rectifying inaccuracies, inconsistencies, and anomalies in large datasets. When data is proven to be trustworthy, it fosters confidence in the information derived from it, ensuring that strategic decisions are based on a solid foundation.

Compliance and Regulations:

In recent years, data privacy regulations such as GDPR (General Data Protection Regulation) and CCPA (California Consumer Privacy Act) have become more stringent, imposing severe penalties for data breaches and privacy violations. Ensuring compliance with these regulations is not just a matter of legal obligation but also a necessity for maintaining customer trust.

Big Data Testing plays a crucial role in ensuring data compliance. It helps identify and rectify sensitive data exposure, data leakage, and other privacy-related issues, protecting the organization from legal repercussions and maintaining customer trust.

Data Quality and Customer Satisfaction

Data quality is directly linked to customer satisfaction. Inaccurate or incomplete customer data can lead to failed transactions, shipping errors, customer service issues, and a host of other problems that can result in lost customers and damage to a company’s reputation.

By rigorously testing and validating big data, organizations can ensure that customer information is accurate and up to date. This not only reduces operational errors but also enhances the overall customer experience.

Business Agility:

In a rapidly changing business environment, the ability to adapt and innovate quickly is a competitive advantage. Big Data Testing helps organizations maintain business agility by ensuring that the data infrastructure is robust and can adapt to changing requirements.

Whether it’s scaling up to handle increased data volumes, integrating new data sources, or adapting to changes in data processing workflows, thorough testing ensures that the system can evolve without disrupting operations.

Demystifying 10 Types of Big Data Testing

Big Data has evolved into a fundamental driver of business growth and innovation. However, its immense volume, variety, velocity, and complexity have brought forth a myriad of challenges, particularly in ensuring the accuracy, reliability, and performance of the data. This is where Big Data Testing comes into play. To understand the nuances of this specialized field, it’s crucial to delve into the various types of testing performed in Big Data Testing. In this article, we will demystify the types of testing involved in the validation of Big Data.

1. Data Ingestion Testing
Data ingestion is the first step in the data processing pipeline, where data is collected from various sources and ingested into the big data system. Data ingestion testing focuses on verifying that the data is collected accurately, ensuring that it’s complete and uncorrupted during the ingestion process. This type of testing helps prevent data loss, corruption, or duplication at the entry point.

2. Data Storage Testing
In the big data ecosystem, data is often stored in distributed file systems and databases. Data storage testing validates that data is correctly stored and retrievable. It involves checking for data completeness, ensuring data partitioning is correct, and verifying data compression techniques. Data storage testing is vital to guarantee the integrity of data throughout its lifecycle.

3. Data Transformation Testing
Data transformation is a critical stage in data processing, where raw data is converted into a format suitable for analysis. This phase involves data cleansing, formatting, and structuring. Testing in this context focuses on ensuring that data transformation processes are applied accurately, without loss of data or alteration of data integrity. It verifies that data is transformed as intended.

4. Data Quality Testing
Data quality testing aims to assess the overall quality of the data. It involves examining data for accuracy, consistency, completeness, and correctness. This type of testing identifies and rectifies issues such as missing values, outliers, and duplicate records. Ensuring data quality is crucial for informed decision-making.

5. Data Integration Testing
Big data environments typically involve data from various sources. Data integration testing validates that data from different sources is correctly integrated into a single repository. It ensures that data relationships and dependencies are accurately maintained, and data flows seamlessly between different components of the big data system.

6. Performance Testing
Performance testing in Big Data involves evaluating the system’s capability to process data efficiently. This includes load testing, stress testing, and scalability testing. Performance testing assesses the system’s ability to handle increasing data volumes, real-time processing, and spikes in workloads without performance degradation.

7. Security and Privacy Testing
In the era of stringent data privacy regulations, security and privacy testing are paramount. It involves ensuring that sensitive data is adequately protected, and access controls are implemented. Security testing also assesses vulnerabilities, while privacy testing verifies compliance with data protection regulations like GDPR and CCPA.

8. Regression Testing
As big data systems are continually evolving, regression testing is crucial to ensure that changes or updates do not introduce new issues or negatively impact existing data quality and performance. It involves retesting to validate that data quality and processing performance remain consistent.

9. User Interface Testing
In cases where big data analytics platforms have user interfaces or dashboards, user interface testing ensures that these interfaces provide a user-friendly experience and accurately represent the data. It focuses on data visualization and usability.

10. Compliance Testing
This type of testing focuses on ensuring that the data processing and storage comply with relevant industry standards and regulations. It assesses whether the system adheres to data governance, auditing, and compliance requirements.

Challenges in Big Data Testing

Data Volume: Big data systems typically involve terabytes or even petabytes of data, making it impractical to test the entire dataset. Testers must select appropriate samples for testing.
Data Variety: Data in big data environments can be structured, semi-structured, or unstructured, making it difficult to define uniform testing strategies.
Data Velocity: The speed at which data is generated and processed is often very high, requiring real-time testing capabilities.
Data Quality: Ensuring data quality is a significant challenge due to the potential presence of dirty data, duplicate records, and missing values.
Scalability: As data volumes grow, the testing infrastructure must be scalable to handle the increasing load.
Complexity and Integration Problems: As big data is collected from various sources, it is not always compatible, coordinated or may not have similar formats as enterprise applications.
Cost Challenges: For a consistent development, integration and testing of big data require for business’s many big data specialist may cost more.
Higher Technical Expertise: Dealing with big data doesn’t include only testers but it involves various technical expertise such as developers and project managers.

Role of Tester In Big Data Testing

Before diving into the role of testers in Big Data Testing, it’s essential to grasp the nuances of big data. Unlike traditional data, big data is characterized by:

Volume: Enormous amounts of data, often measured in terabytes, petabytes, or even exabytes.
Variety: Data comes in diverse formats, including structured, semi-structured, and unstructured data.
Velocity: Data is generated, ingested, and processed at high speeds, sometimes in real-time.
Complexity: Data sources can be distributed, and processing can involve multiple technologies and systems.

The Tester’s Role in Big Data Testing

1. Test Planning and Strategy
Testers are responsible for creating a comprehensive Big Data Testing strategy. This includes understanding the objectives of the testing process, defining testing criteria, selecting appropriate testing methodologies, and establishing a clear roadmap for the testing process.

2. Data Validation
One of the fundamental responsibilities of testers in Big Data Testing is to validate the data. This involves creating validation rules and algorithms to verify data correctness and consistency. Testers must ensure that data adheres to the defined standards and requirements.

3. Data Quality Assurance
Testers play a pivotal role in ensuring data quality. They need to identify and address issues related to data accuracy, consistency, completeness, and integrity. This involves detecting and rectifying anomalies, missing values, duplicate records, and other data quality issues.

4. Performance Testing
Performance testing is crucial in the world of big data, where data processing speed can impact real-time decision-making. Testers assess the system’s ability to handle varying workloads, scalability, and efficiency. This includes load testing, stress testing, and performance optimization.

5. Data Security and Privacy
Ensuring data security and privacy is a paramount concern, particularly in light of stringent data protection regulations. Testers must validate that sensitive data is adequately protected, access controls are in place, and data is anonymized or masked to maintain privacy.

6. Regression Testing
Big data systems are constantly evolving. Testers are responsible for conducting regression testing to ensure that new changes or updates do not introduce new issues or negatively impact existing data quality and performance.

7. Collaboration and Communication
Testers collaborate closely with data engineers, data scientists, quality assurance teams, and business stakeholders to ensure alignment on testing objectives and results. Effective communication is essential for a successful Big Data Testing process.

8. Selecting and Using Testing Tools
Testers choose and utilize a range of tools and frameworks designed for Big Data Testing. Tools like Apache Hadoop, Apache Spark, and JMeter help automate testing processes, analyze data, and validate data processing efficiency.

9. Compliance Testing
In a world governed by data privacy regulations, testers play a crucial role in ensuring that the big data system complies with relevant industry standards and legal requirements. They conduct compliance testing to verify adherence to data governance, auditing, and compliance mandates.

Benefits of Big Data Testing

Big Data has transformed the way organizations operate and make informed decisions. The wealth of information it provides can be a game-changer, but only if it’s reliable and accurate. This is where Big Data Testing comes into play. By systematically validating and ensuring the quality of data, Big Data Testing offers a multitude of benefits that are crucial for data-driven success. In this article, we’ll explore the advantages of Big Data Testing.

1. Data Accuracy
One of the primary benefits of Big Data Testing is the assurance of data accuracy. Ensuring that the data is correct, consistent, and trustworthy is fundamental to making informed decisions. By rigorously testing data, organizations can rely on the accuracy of the insights derived from it, reducing the risk of erroneous decisions and costly mistakes.

2. Enhanced Data Quality
Big Data Testing is instrumental in enhancing data quality. It helps identify and rectify issues such as missing values, duplicate records, outliers, and inconsistencies. By maintaining high data quality standards, organizations can provide superior customer experiences, streamline operations, and ensure the integrity of their data assets.

3. Improved Decision-Making
Reliable and accurate data is the bedrock of effective decision-making. With Big Data Testing, organizations can have confidence in the data they rely on for strategic choices. This leads to more informed decisions, better resource allocation, and the ability to spot opportunities and threats in a timely manner.

4. Data Privacy and Compliance
In a world where data privacy regulations are increasingly stringent, Big Data Testing helps organizations ensure they are in compliance. By identifying and mitigating potential data privacy and security vulnerabilities, organizations can avoid legal repercussions and maintain customer trust.

5. Optimized Performance
Performance testing in Big Data ensures that data processing jobs are efficient and timely. By assessing the system’s ability to handle varying workloads and optimizing its performance, organizations can make real-time decisions with confidence and agility.

6. Data Scalability
As data volumes continue to grow, the scalability of data systems becomes paramount. Big Data Testing helps assess and confirm that the system can handle increasing data loads effectively. It ensures that organizations can grow their data infrastructure without compromising on performance.

7. Business Agility
In a rapidly changing business environment, the ability to adapt quickly is a competitive advantage. Big Data Testing supports business agility by ensuring that the data infrastructure is robust and adaptable. It enables organizations to respond to changing requirements, integrate new data sources, and evolve their data processing workflows without disruption.

8. Cost Reduction
By identifying and rectifying data quality issues and optimizing performance, Big Data Testing can lead to cost reductions. It minimizes the operational errors that can result from inaccurate data, streamlines data processing, and prevents costly compliance violations.

9. Customer Satisfaction
High-quality data ensures that customer information is accurate and up to date. This not only reduces operational errors but also enhances the overall customer experience, leading to increased customer satisfaction and loyalty.

10. Maintaining Trust
Ultimately, Big Data Testing helps organizations maintain the trust of their stakeholders. When data is proven to be reliable and accurate, it fosters trust and confidence in the information derived from it, both internally and externally.

Big Data Testing Tools

Several tools and frameworks are available to aid in Big Data Testing. Some of the most popular ones include:

Apache Hadoop: A framework that supports distributed storage and processing of big data. Hadoop can be used for testing data processing and storage components.
Apache Spark: An open-source, distributed computing system, widely used for real-time data processing and analytics. Spark can be employed for performance testing.
Apache Kafka: A distributed streaming platform that is invaluable for testing real-time data ingestion and processing.
Hive: A data warehouse infrastructure that can be used for data validation and querying.
Selenium: While typically associated with web application testing, Selenium can also be employed for testing big data user interfaces and dashboards.
JMeter: A popular tool for load and performance testing, JMeter can be be used to evaluate the performance of big data systems.

Big Data Testing Best Practices

To overcome the challenges associated with Big Data Testing, it’s essential to follow best practices. Here are some of the key guidelines for effective Big Data Testing:

Define Clear Testing Objectives: Clearly define the testing objectives, including what aspects of data quality and performance are critical to your organization.
Data Sampling: Instead of testing the entire dataset, use statistical sampling techniques to select representative data subsets for testing.
Automation: Leverage automation tools to design and execute test cases, as manual testing can be time-consuming and error-prone.
Data Validation: Develop validation rules and algorithms to verify data correctness and consistency.
Data Masking: Ensure sensitive data is masked or anonymized to maintain data privacy and security.
Performance Testing: Assess the performance of data processing jobs, as latency can severely impact real-time decision-making.
Scalability Testing: Validate whether the system can handle increased data loads and scaling demands effectively.
Regression Testing: Continuously test data quality and performance as data sources and data processing logic evolve.

Afterthought?

Big Data Testing is a critical component in ensuring the reliability, accuracy, and performance of big data systems.

As organizations increasingly rely on data to make informed decisions, the importance of effective Big Data Testing cannot be overstated.

By following best practices, leveraging the right tools, and embracing automation, organizations can navigate the challenges of testing large and complex datasets and ultimately gain confidence in their data-driven strategies.

30 minutes IoT Testing tutorial for beginners

Rahul — Tue, 11 Sep 2018 12:29:15 +0000

Internet of Things ! One of the most rapidly growing “thing” in Business and technology space is “Internet of things” or IoT.

The Internet of Things (IoT) has transformed the way we interact with technology and the world around us. From smart thermostats and wearable fitness trackers to connected cars and industrial sensors, IoT devices are all around us. However, ensuring the reliability, security, and functionality of these devices is essential. This is where IoT testing comes into play. In this tutorial, we will walk you through the fundamentals of IoT testing, making it accessible for beginners.

So let’s see what IoT is all about followed by its testing aspect.

What’s IOT all about?

IoT encompasses a wide range of interconnected devices that collect, transmit, and process data. These devices can be as simple as a temperature sensor or as complex as a self-driving car. IoT testing involves evaluating the functionality of these devices and ensuring they work seamlessly within the larger ecosystem.

Overview of the top-level components

Stage 1. Sensors/actuators

Sensors: These are the devices that collect data from the physical environment. Sensors can measure a wide range of parameters such as temperature, humidity, light, motion, pressure, and more. They act as the input devices of the IoT system, gathering data from the real world.
Actuators: Actuators, on the other hand, are responsible for taking actions based on the data collected by sensors. They can control physical processes, devices, or machinery. For example, actuators can turn on a fan when a temperature sensor detects that a room is too hot. Actuators are the output devices of the IoT system.

Stage 2. Network / Gateway

Network Protocols: This layer handles the communication between IoT devices, sensors, and actuators. IoT devices use various communication protocols such as Wi-Fi, Bluetooth, Zigbee, LoRaWAN, and cellular (3G, 4G, 5G) to transmit data and commands. These protocols ensure that devices can send and receive information reliably.
Gateways: In many IoT scenarios, a gateway device acts as an intermediary between IoT devices and the cloud or central server. Gateways collect data from sensors, preprocess it, and transmit it to the cloud. They can provide protocol translation, security, local processing, and even serve as a point of aggregation for data from multiple sensors.

Stage 3. Service / data processing layer

In IoT architecture, there isn’t a distinct “Service/Data Processing Layer” in the same way that there are clearly defined layers like Sensors/Actuators, Network/Gateway, and Application layer. However, data processing and services are integral parts of an IoT system and are typically handled within the Application layer.

Here’s how data processing and services are typically incorporated into the IoT architecture:

Data Processing:
- Data processing in IoT occurs within the Application layer. This layer is responsible for receiving, storing, and analyzing the data collected from sensors and devices.
- Data processing tasks include data filtering, aggregation, transformation, and analytics. These processes help derive meaningful insights from raw sensor data.
- Data can be processed in real-time (e.g., for immediate alerts and responses) or in batch mode (e.g., for historical analysis).
Services:
- IoT services, which are often a part of the Application layer, encompass a range of functionalities and capabilities that make IoT systems useful to end-users.
- Services can include applications for home automation, industrial control, healthcare monitoring, predictive maintenance, and more.
- Services may also provide features like remote device management, firmware updates, and security measures.
- User interfaces (UIs) that allow end-users to interact with IoT devices and control them are considered IoT services.

Stage 4. Application layer (Includes Device)

Applications: The Application layer is where IoT data is processed, analyzed, and put to practical use. IoT applications include software and services that interpret the data collected by sensors and provide value to end-users. These applications can range from smart home systems and industrial automation solutions to healthcare monitoring platforms.
User Interfaces: Within the Application layer, user interfaces (UIs) enable end-users to interact with IoT systems. UIs can be web-based dashboards, mobile apps, voice-activated assistants, or other interfaces that allow users to control and monitor IoT devices.

Internet of Things Testing Important?

IoT devices often perform critical tasks, such as monitoring patient health in healthcare or controlling manufacturing processes in industries. A failure in an IoT device can lead to serious consequences. Testing is essential to identify and rectify issues before they become problems.

Technology used in IoT

IoT (Internet of Things) relies on a variety of technologies and components to enable the connectivity, data collection, and communication that characterize IoT devices. Here are some key technologies used in IoT:

Sensors and Actuators: These are fundamental components of IoT devices. Sensors collect data from the physical environment, while actuators allow IoT devices to interact with the physical world. Examples include temperature sensors, motion detectors, and motors.
Connectivity Protocols:
- Wi-Fi: Commonly used for indoor applications.
- Bluetooth: Often used for short-range connections, such as in wearables or home automation devices.
- Cellular (3G, 4G, 5G): Provides wide-area coverage for mobile IoT devices.
- LoRaWAN: A low-power, long-range wireless protocol for IoT applications.
- Zigbee and Z-Wave: Wireless protocols for low-power, short-range communication in home automation.
- NFC (Near Field Communication): Used for very short-range communication, such as contactless payments.
Embedded Systems: These are the microcontrollers and processors that power IoT devices. They manage data processing, device communication, and often include firmware for device operation.
Cloud Computing: Cloud platforms play a crucial role in IoT by providing storage, data analysis, and scalability. Services like AWS IoT, Google Cloud IoT, and Azure IoT Hub are popular choices for IoT deployments.
Data Analytics and Machine Learning: IoT devices generate massive amounts of data. Analytics tools and machine learning algorithms are used to extract meaningful insights from this data, enabling predictive maintenance, anomaly detection, and more.
Edge Computing: In some cases, data processing is performed closer to the source (at the edge) rather than in the cloud. Edge computing reduces latency and can be essential for real-time applications.
Blockchain: Blockchain technology is used to enhance the security and trustworthiness of IoT data and transactions, particularly in scenarios where data integrity is critical.
Security Solutions: IoT devices are vulnerable to security threats. Technologies like encryption, secure boot, and security standards (e.g., IoT Security Foundation) are used to protect IoT ecosystems.
RFID (Radio-Frequency Identification): RFID technology is used for tracking and managing assets. It’s commonly used in logistics and supply chain management.
MQTT and CoAP: These are lightweight messaging protocols designed for IoT communication. They facilitate efficient data exchange between devices and servers.
Operating Systems: IoT devices often run specialized operating systems designed for resource-constrained environments. Examples include Linux-based OSs, FreeRTOS, and Zephyr.
Geolocation Technologies: GPS and other geolocation technologies are used in IoT to track the location of devices and assets.
Fog Computing: This extends the concept of edge computing by adding more computing resources in proximity to IoT devices. It’s especially useful for applications that require real-time processing.
Voice and Speech Recognition: IoT devices like smart speakers and voice-controlled appliances rely on voice recognition technology, such as Amazon Alexa or Google Assistant.
Databases: Databases, both traditional and NoSQL, are used for storing and managing IoT data efficiently.
RF (Radio Frequency) Technologies: RF technologies like RFID, NFC, and UHF are used for wireless communication and identification in IoT devices.
AI Processors: Specialized AI hardware accelerators are integrated into some IoT devices to enable machine learning and artificial intelligence at the edge.

IoT Testing framework

Testing IoT Systems

Defining test cases for IoT devices can be considered as a uphill task. Other than testing the real life scenarios there are a few common test scenarios you need to consider while testing IoT devices and the network.

1. Functional Testing

Includes the testing of all functional use cases of IoT application which also includes user experience and usability testing.

This focuses on verifying if the IoT device performs its intended functions correctly. It includes testing sensor data accuracy, actuator control, and device communication.

Example: Verify IoT application has all required features working as per the specifications or verify whether the User Experience (UX) is up to the mark.

2. Usability Testing

3. Connectivity

The usability of IoT devices is also an important aspect to consider while testing.

Here are a few usability test cases for the scenario of using a smartwatch to make NFC (near field communication) payments with bank.

• Time is taken in a transaction.
• How quickly a user can place transaction.
• Payment can be made only on authenticated NFC enabled POS machines.
• If the wearable is lost, the user should have the provision to block the device in pre-defined time frame.

The success of an IoT system depends on how well the devices and hub are connected.

Below are some example tests to verify the Connectivity:

• Regular ping messages should be sent by the device to make sure the connection is not lost.
• Verify that gadget transmits keep-alive message in a regular interval.
• Sending user a notification, while operating in offline mode, makes your service reliable.
• Verify that IoT gadget need to inform the network about power status.

4. Security Testing

5. Interoperability Testing

With IoT devices being prone to cyber threats, security testing ensures that data transmission is encrypted, devices are protected from unauthorized access, and vulnerabilities are identified and fixed.

Here are a few examples of possible tests:

• Keep User Interface of the software secure from unauthentic logins by using a strong password.
• Proper authentication before communication starts. For example, in case of Bluetooth connection, only paired devices should be able to communicate.
• Establishment of the data connection post successful registration.

This testing validates the connectivity across all the devices and protocols in the IoT set up.

Interoperability Testing in the Service Layer of the IoT framework becomes important as IoT standards and specifications require platforms to be communicable and operable across devices, regardless of make, model, manufacturer or industry.

6. Performance Testing

7. Compatibility Testing

This examines the responsiveness and stability of IoT devices under varying conditions, such as heavy loads, weak network connectivity, or extreme temperatures.

Below can be a good starting point for Performance tests:

• Connected device should be able to send any amount of data (“Any” Amount of data should be as defined).
• Re-initiation of data transfer if data sent by the device exceeds a predefined amount,
• Ability of data transfer in case of low power / battery status of device.

Since IoT devices often need to work together, compatibility testing checks if different devices and protocols can interact smoothly.

The software should support numerous devices and should know which nodes should be preferred while developing connections.

If a user needs to make a payment using IoT software, it should be capable of a transaction through numerous banks.

Tools for IoT software testing

Shodan
Shodan is a connectivity testing tool that verifies the devices connected to the hub. It shows the connected devices, their location, and information of its user. It keeps a record of all the computers connected to the network that are either directly or indirectly connected to the internet.
MQTT Spy
MQTT Spy is a useful tool if your device supports MQTT protocol. It is one of the most efficient open source packages available for IoT Testing and is specifically helpful for people with day-to-day data usage.
Wireshark
Wireshark is an opensource application that lets you monitor the traffic, host addresses, protocols.
TCPDump
This application performs the similar jobs as Wireshark with an exception that TCPDump doesn’t have a User Interface. It is a command-line packet analyzer that also monitors the traffic i.e. displaying the TCP/IP and other packets that are transmitted over a network.
JTAG Dongle: This is similar to a debugger in PC applications. This helps in debugging the target platform code and show variable step by step.
Digital Storage Oscilloscope: This is used to check various events with time stamps, glitches in power supply, signal integrity check.
Software Defined Radio: This is used to emulate receiver and transmitter for a large range of wireless gateways.
Robot Framework: A versatile open-source automation framework.
Cypress: A powerful end-to-end testing framework.
Postman: For testing IoT APIs.
Selenium: Ideal for web-based IoT applications.

Afterthought?

This is just a beginning. While the IoT brings a different level of complexities to testing, the business opportunities it has revealed is tremendous.

IoT testing is a vital aspect of ensuring the reliability, security, and performance of IoT devices. With the growing ubiquity of IoT, learning how to test these devices is a valuable skill. By following the steps and best practices outlined in this tutorial, beginners can build a solid foundation in IoT testing and contribute to the quality and safety of the IoT ecosystem. Happy testing!

Don’t leave with complexities of IoT. Brenden – A born artist and noted cartoonist from Auckland has created an IoT imagination below. Visit cartoonsbyjim.com for more amazing creations.

A step by step Blockchain testing guide for beginners

Rahul — Sun, 02 Sep 2018 11:11:24 +0000

A step by step Blockchain testing guide for beginners

A bit of Bitcoin – Almost everyone might have heard of the crypto currency – Bitcoin which has seen a tremendous growth and movement. A research by the University of Cambridge estimates that in 2017, there were 2.9 to 5.8 million unique users using a crypto currency wallet, most of them using Bitcoin, think about how many users might have been added since then!

Okay what about blockchain – Block chain is the technology on which Bitcoin is built on. The first blockchain was conceptualized by Satoshi Nakamoto in 2008 which was implemented the following year by Nakamoto as a core component of the crypto currency bitcoin. Since then, Blockchain has been gaining popularity and has become a buzzword.

Blockchain technology has gained immense popularity in recent years for its potential to disrupt various industries. From finance to healthcare, the applications of blockchain are vast and promising. However, before any blockchain-based project goes live, thorough testing is essential to ensure its security, functionality, and reliability. This guide is aimed at beginners, providing a step-by-step approach to blockchain testing.

With the rise of popularity and application of Blockchain technology, testing of Blockchain is also coming up and industry experts believe that it has a potential of becoming a “Hot cake” in the coming years. Who doesn’t want to eat it!

Okay, let’s dive a little deeper into what exactly is blockchain technology, how it can help us, and then what essential aspects to test in Blockchain.

What is Blockchain Technology?

In simple words, a blockchain is a record of transactions, like a traditional ledger. These transactions can be any movement of money, goods or secure data—a purchase at a supermarket.

Blockchain technology is a decentralized and distributed ledger system that underlies most cryptocurrencies, including Bitcoin. It is a revolutionary and disruptive technology that has the potential to impact a wide range of industries beyond finance. At its core, a blockchain is a digital, tamper-proof record-keeping system that allows multiple parties to maintain a shared database without the need for a central authority.

Here are some key aspects of blockchain technology:

Decentralization: Instead of relying on a central authority, such as a bank or government, a blockchain operates on a network of computers (nodes) that work together to validate and record transactions. This decentralization makes it resistant to censorship and control by any single entity.
Distributed Ledger: A blockchain is a distributed ledger that stores data across a network of nodes. Each node has a copy of the entire blockchain, and they work collectively to validate and record new transactions.
Immutability: Once data is added to a blockchain, it is extremely difficult to alter or delete. This immutability is achieved through cryptographic hashing and consensus mechanisms.
Transparency: Blockchain transactions are visible to all participants in the network. This transparency enhances trust and accountability, as anyone can audit the transaction history.
Security: Blockchain uses advanced cryptographic techniques to secure data. Transactions are grouped into blocks, and each block contains a reference to the previous block, creating a chain. This makes it very challenging for anyone to alter a transaction without changing all subsequent blocks.
Consensus Mechanisms: Blockchains rely on consensus mechanisms to agree on the validity of transactions. For example, Bitcoin uses Proof of Work (PoW), while other blockchains use different methods like Proof of Stake (PoS) or Delegated Proof of Stake (DPoS).
Smart Contracts: Smart contracts are self-executing contracts with the terms of the agreement directly written into code. They can automatically execute when predefined conditions are met.
Use Cases: Blockchain technology has applications beyond cryptocurrencies, including supply chain management, voting systems, identity verification, and healthcare records. It is often used to create trust in situations where intermediaries are traditionally required.
Challenges: Blockchain technology faces challenges such as scalability, energy consumption (for PoW-based blockchains), and regulatory issues. These challenges are actively being addressed by ongoing research and development efforts.

Types of blockchains

Currently, there are three types of blockchain networks – public blockchain, private blockchain and consortium blockchain.

Public blockchain: A public blockchain has absolutely no access restrictions. Anyone with an internet connection can send transactions to it as well as become a validator. Some of the largest, most known public blockchain are Bitcoin and Ethereum.
Private Blockchain: A private blockchain is permissioned. One cannot join it unless invited by the network administrators. Participant and validator access is restricted. This type of blockchain can be considered a middle-ground for companies that are interested in the blockchain technology in general but are not comfortable with a level of control offered by public networks. Typically, they seek to incorporate blockchain into their accounting and record-keeping procedures without sacrificing autonomy and running the risk of exposing sensitive data to the public internet.
Consortium blockchain: A consortium blockchain is often said to be semi-decentralized. It, too, is permissioned but instead of a single organization controlling it, a number of companies might each operate a node on such a network. The administrators of a consortium chain restrict users’ reading rights as they see fit and only allow a limited set of trusted nodes to execute a consensus protocol.

Key terms in Blockchain world:

Block is a piece of code that contains a list of transactions. The first block in the chain is called the genesis block.
Blockchain is a constantly growing chain of blocks. The copies of the chain are stored on a number of computers (nodes) that partake in the network.
Crypto currency is digital money with no physical equivalent.
Bitcoin is a digital payment method and the most popular kind of crypto currency these days.
Ethereum is the second most popular crypto currency with the large market capitalization (second only to Bitcoin).
Fork is a change to the blockchain protocol that results in a chain split into two chains that will function independently.
Mining is the process by which transactions are verified and added to the blockchain, and also the means through which new Bitcoin are released.
Fee is the commission miners get for verifying a transaction and adding it to the blockchain.
Faucets are websites that give away small portions of Bitcoin for free
Smart contract are software modules on the blockchain that automatically execute transactions based on pre-defined conditions and business logic.

What is Blockchain Testing?

Blockchain testing is the process of systematically evaluating the various components and functionalities of a blockchain-based application to ensure its correctness, security, and performance. This includes testing smart contracts, performance under varying loads, security vulnerabilities, and how well it integrates with other systems.

What is the need for testing in a Blockchain?

Blockchain applications are often associated with real-world assets and significant financial transactions. Any flaws or vulnerabilities in these applications can have severe consequences. Testing is crucial to prevent issues like smart contract vulnerabilities, data breaches, and unreliable performance.

A block once added to the blockchain remains there forever and if you try to change the data in some block in between the chain, the following blocks become invalid. A single change in block of the blockchain will cause every subsequent blocks to change as well. This makes it important that whenever a new block is added, it’s being added the right way because it cannot be changed at a later date. It becomes complex to exploit a blockchain and the testing of blockchain becomes even more complex.

Add to that, it’s contributes to large transactions which goes through validation, encryption, decryption, transmission so it becomes necessary to make sure that these processes go smoothly.

Setting Up Your Test Environment

Selecting the Right Blockchain Platform
Before you start testing, choose the appropriate blockchain platform for your project. Popular choices include Ethereum, Binance Smart Chain, or Polkadot. The choice depends on your project’s requirements.

Choosing the Appropriate Development Tools
Select the development tools and frameworks that best fit your platform. Tools like Truffle, Remix, or Solidity IDEs can help streamline smart contract testing and development.

Setting Up a Local Testnet
Set up a local blockchain network for testing using tools like Ganache or Geth. This allows you to conduct tests without incurring the cost associated with the mainnet.

What and how do we test in a Blockchain?

Smart Contract testing:
Smart Contracts are software modules on the blockchain that automatically execute transactions based on pre-defined conditions and business logic. Testing smart contracts involves simulation of all possible expected and unexpected conditions for every contract, testing all combinations of business logic and the proper triggering and correct execution of transactions.

1.1 Writing Test Cases
Develop comprehensive test cases that cover different scenarios. This should include unit tests, integration tests, and end-to-end tests to ensure the smart contracts operate correctly.

1.2 Deploying Smart Contracts
Deploy your smart contracts to the test network, allowing you to interact with them as you would in a real environment.

1.3 Testing for Security Vulnerabilities
Conduct thorough security testing to identify vulnerabilities such as reentrancy attacks, overflows, or permission issues.

1.4 Functional Testing
Verify that your smart contracts function as intended, including transaction processing and data storage.
Performance Testing

2.1 Understanding Performance Metrics
Identify key performance metrics, such as transaction throughput, confirmation times, and block propagation, to benchmark the performance of your blockchain application.

2.2 Load Testing
Simulate various loads to evaluate how your blockchain handles different transaction volumes. This helps in understanding the limits and bottlenecks of your system.

2.3 Stress Testing
Subject your blockchain to extreme conditions to test its resilience and behavior under duress. This can help uncover potential vulnerabilities.
Security Testing

3.1 Vulnerability Assessment
Regularly scan for vulnerabilities using specialized tools and conduct code reviews to ensure that your codebase is secure.

3.2 Penetration Testing
Employ ethical hackers to attempt to breach your system’s security, identifying weaknesses that need to be addressed.

3.3 Code Review
Examine the smart contract code to ensure it adheres to best practices and security standards.
Integration Testing

4.1 API Testing
Test the interactions of your blockchain application with other systems through APIs.

4.2 Cross-Platform Compatibility
Ensure that your blockchain application can work seamlessly with various platforms and devices.

4.3 Interoperability Testing
If your blockchain interacts with other blockchains or systems, verify that this integration works as intended.
User Acceptance Testing

5.1 Creating User Scenarios
Invite actual users to interact with your blockchain application and gather feedback on the user experience.

5.2 Feedback Gathering
Collect and analyze user feedback to make necessary improvements based on real-world usage.
Block Size and Chain Size Testing:

Block Size refers to the maximum size of data that can be included in a single block, while Chain Size pertains to the cumulative size of the blockchain. Testing these aspects is essential to maintain efficiency and scalability.

6.1 Load Testing:
Create various scenarios to test how the network handles different block sizes. Incrementally increase the block size and observe network performance and confirmation times.

6.2 Stress Testing:
Stress the blockchain with a high volume of transactions to assess how it handles an ever-growing chain. Measure synchronization times and resource consumption on nodes.
Peer/Node Testing:

Testing the interaction between nodes (peers) is crucial to verify that the network is functioning correctly and securely.

7.1 Node Connectivity Testing: Check how well nodes can discover and connect to each other. Ensure nodes maintain consistent connections, even in the presence of network disruptions.

7.2 P2P Protocol Testing: Validate that the peer-to-peer communication protocol is correctly implemented and secure. Test for the ability to propagate new blocks and transactions among peers.
Cryptographic Data Testing:

Cryptographic integrity is fundamental to blockchain security. You should ensure that cryptographic data, such as signatures and hashes, is accurate.

8.1 Signature Verification: Verify that digital signatures on transactions and blocks can be correctly validated. Create test cases with both valid and invalid signatures.

8.2 Hash Function Testing: Ensure that the cryptographic hash functions used in the blockchain are resistant to collisions and pre-image attacks.
Consistency Testing:

Consistency in a blockchain refers to the uniformity of data across all nodes. Inconsistencies can indicate potential issues.

9.1 State Consistency Testing: Check whether the state of the blockchain is consistent among all nodes. Test for correct balances, contract storage, and execution results.

9.2 Fork and Reorganization Testing: Create scenarios that trigger forks and reorganizations in the chain. Ensure that the network can handle these situations and that consensus is maintained.
Data Corruption Testing:

Data corruption can lead to catastrophic failures in a blockchain. Testing for data integrity is crucial.

10.1 Data Corruption Scenarios: Inject corrupted data into the blockchain and observe how the network responds. Ensure that the blockchain can detect and reject corrupted blocks.

10.2 Data Recovery Testing: Test the ability of the network to recover from data corruption by using backups or other mechanisms.

Blockchain testing frameworks

There are several blockchain testing frameworks and tools available to help streamline the testing process for blockchain applications. These frameworks offer various features for testing different aspects of blockchain projects, including smart contracts, network performance, and security. Here are some popular blockchain testing frameworks:

Truffle:
Truffle is one of the most widely used blockchain testing frameworks for Ethereum-based projects. It provides a suite of tools for developing and testing Ethereum smart contracts. Truffle allows you to write comprehensive test cases using JavaScript and deploy them to the Ethereum test network or a local testnet.
Ganache:
Ganache, part of the Truffle suite, is a personal blockchain emulator that makes local testing of Ethereum smart contracts easy. It provides a local testnet for quick development and testing, allowing you to simulate various network conditions and scenarios.
Hardhat:
Hardhat is another Ethereum-focused development and testing framework. It offers a robust environment for developing, compiling, deploying, and testing smart contracts. Hardhat is known for its extensibility and developer-friendly features.
Embark:
Embark is a framework for Ethereum decentralized applications (dApps) that includes a testing suite. It supports writing and running JavaScript and Solidity test cases and provides an integrated development environment for Ethereum development.
Populus:
Populus is a Python-based development and testing framework for Ethereum smart contracts. It enables developers to write test cases using Python and is known for its simplicity and ease of use.
Truffle Teams:
Truffle Teams is a cloud-based platform that offers continuous integration and continuous deployment (CI/CD) for Ethereum smart contracts. It provides an integrated testing environment and supports automated testing.
Cypress:
While not blockchain-specific, Cypress is a popular end-to-end testing framework that can be used to test blockchain applications’ front-end interfaces. It offers features like automated browser testing, which can be valuable for dApps.
Mocha and Chai:
Mocha is a widely used JavaScript testing framework, and Chai is an assertion library. Together, they can be used to write test cases for Ethereum smart contracts. They are versatile and can be used alongside other Ethereum testing tools.
Ethers.js:
Ethers.js is a JavaScript library for interacting with Ethereum. It can be used to write test cases for smart contracts and interact with the Ethereum blockchain programmatically.
Mythril and MythX:
Mythril is a security analysis tool for Ethereum smart contracts. MythX is a cloud-based service that integrates with Mythril to provide in-depth security analysis. These tools help identify vulnerabilities in smart contracts and can be integrated into your testing process.

A simple test strategy across the test phases

Recap

Block chain is the technology on which Bitcoin is built on.
Blockchain is a record of transactions, like a traditional ledger. These transactions can be any movement of money, goods or secure data—a purchase at a supermarket.
Blockchain provides a solution to the existing issues of delay and dependency on one provider during a transaction through the validation mechanism called as Proof of block and decentralization of validating authority.
Critical aspects to test in Blockchain are to test Block size, data size, Smart Contracting, Load, security and data transmission.
Ethereum Tester, BitcoinJ and Populus are the main testing framework for Blockchain testing.

Afterthought

Since quality cannot be an afterthought, it needs to pro-actively built-in, practicing ‘value by design’ principles.

Blockchain testing is a critical phase in the development of any blockchain-based project.

While the overall testing process and test phases are same as testing any other application, there are some notable differences as Blockchain technology contains some peculiar and critical components.

Beginners should follow a structured approach, as outlined in this guide, to ensure that their blockchain applications are secure, performant, and reliable. Consistent testing and continuous improvement are key to success in the world of blockchain technology.