api functional testing https://www.softwaretestingportal.com Let's learn something new.....together Tue, 16 Jun 2020 00:26:09 +0000 en-US hourly 1 https://wordpress.org/?v=7.0.6 https://www.softwaretestingportal.com/wp-content/uploads/2018/09/cropped-Logo-again-middle-1-1-32x32.jpg api functional testing https://www.softwaretestingportal.com 32 32 20 API Testing Tools – Beyond SOAP UI and Postman https://www.softwaretestingportal.com/api-testing-tools/ https://www.softwaretestingportal.com/api-testing-tools/#comments Fri, 10 Apr 2020 01:36:06 +0000 http://www.softwaretestingportal.com/?p=992

We all know about API testing well. In my previous article – API testing, Key Terminologies and more……we learnt about the basics of API testing. Let’s now take a look on some of the API Testing tools.

To learn something new, I haven’t considered the leaders like SOAP UI, Postman, JMeter. Rather I have explored some tools which may be lesser known but have more or less same features and give a solid performance therefore worth giving a shot.

#1 ACCELQ

  • Zero code API Test Automation on Cloud.
  • Support for leading technologies.
  • API and UI Test Automation in the same simplified flow.
  • CI Driven regression suite executions.
  • Dynamic Environment management.
  • Directly correlate Business process and corresponding API for complete coverage.
  • API Test Case Management, Test Planning, Execution and tracking governance.
  • Trial – 14 days free trial

#2 API Fortress

  • API Fortress is a REST & SOAP API, Web Services, and Microservices testing platform.
     Out-of-the-Box Integrations.
  • Deploy On-Premises or in the Cloud.
  • Quickly Generate and Edit Functional Tests
    Detailed Reporting.
  • 30 days free trial

#3 API Science

  • API Science is an advanced API Monitoring service.
  • Monitor own APIs
  • Multi-step REST transactions
  • Advanced tests & validations
  • Get alerted on any issue
  • Run tests at any time
  • Monitor your own API & 3rd party APIs
  • Customized reporting
  • Intelligent API diagnostics
  • Collaborate using
  • team plans

#4 Apigee

  • Easily create API proxies and visually configure or code API policies as steps in the API flow.
  • Enforce consistent security best practices and governance policies across all APIs.
  • Publish APIs by providing reference documentation, manage the audience for an API from portal.
  • Analyze APIs, API traffic data, investigate traffic spikes, and trace live API calls.
  • Monitor APIs to ensure API availability.
    Publish APIs to a customizable portal, so that developers can easily explore, test, get API keys, and innovate fast.
  • 60 days free trial.

#5 Assertible

  • Can automatically sync API tests with the latest changes in the specification.
  • Integrates with the tools e.g. push code to GitHub or send alerts to Slack if failures happen.
  • Easily create tests for web service.
  • Continuously test multiple deployment and environment.
  • Set up scheduled monitoring to continuously test the services.
  • Run the same tests on multiple environments.
    Automatically execute post-deploy smoke tests.
  • HTTP assertions allow you to create domain specific test.

#6 Citrus Framework

  • Citrus is open source and licensed under Apache License 2.0.
  • Test automation by creating repeatable integration tests within continuous build.
  • Test complex messaging scenarios with step by step sequence of operations.
  • Use advanced logic in test cases including loops, delays, retries, parallel sections.
  • Json message payload validation.
  • JsonPath element validation.
  • Groovy Json validation.
    Simple test creation.
  • IDE support – execute tests as TestNG/JUnit tests from Eclipse, IntelliJ IDEA or NetBeans.

#7 HTTPMASTER

  • Development and test tool for REST web services and API applications.
  • Can display and validate the most common REST formats; XML, JSON, and HTML
  • Broad set of http properties.
  • Response data validation
    Simulate variations of input data or create global values for reuse across multiple API calls.
  • Enables to define chaining items of different types.
  • Support for data upload which enables users to post plain text data, contents of file, or mix of plain text data and disk files.
  • Free 30-day trial

#8 Insomnia

  • Free and open source on Mac, Windows, and Linux.
  • Reuse API keys or session IDs. Define environment variables globally or switch between sub-environments.
  • Generate http code for over thirty language libraries, including Curl, NodeJS, Go, Swift, Python, Java, C.
  • Free version available.

#9 Mockbin

  • Mockbin allows you to generate custom endpoints to test, mock, and track HTTP requests & responses between libraries, sockets and APIs.
  • Mock custom endpoints using any HTTP Archive (HAR) response object.
    Test using your preferred language.
  • Mockbin accepts all standard Methods and allows method overriding in addition to Get and Post.
  • Mockbin supports output in JSON, YAML and XML, as well as an HTML.
  • Log and inspect incoming calls to your custom endpoints.

#10 Ping API

  • Ping-API is API testing allows to write test script in JavaScript and CoffeeScript.
  • It allows inspecting HTTP API call with a complete request and response data.
  • Can schedule test in every minutes or hour
    Support for writing script to set request headers, body and URL parameters.
  • It also supports validation of response headers and body.
  • Easy to generate test script with the input of API parameters.
  • Free for limited projects, varied pricing afterwards.

#11 Postwoman


Postwoman is an open-source API request builder. It helps you create your requests faster, saving you precious time on your development.

It also helps to create the requests faster, saving precious time on overall development.

  • Free, Open Sourced
  • Runs online
  • Self Hosted
  • Multi-platform & multi-device support
  • Progressive Web App
  • WebSocket testing
  • Customizations allowed
  • Pre-Request Scripts and environment variables setup
  • Docker support

#12 pyresttest

  • A REST testing and API microbenchmarking tool.
  • Tests are defined in basic YAML or JSON config files, no code needed.
  • Minimal dependencies (pycurl, pyyaml, optionally future), making it easy to deploy on-server for smoketests/healthchecks.
  • Supports generate/extract/validate mechanisms to create full test scenarios.
  • Returns exit codes on failure, to slot into automated configuration management/orchestration tools (also supplies parseable logs).
  • Logic is written and extensible in Python.
  • Return exit codes for failed results.
  • The building of test scenarios with generate/extract/validates mechanisms.
  • Easy deployment on the server.

#13 ReadyAPI

ReadyAPI is an integrated suite of API testing tools that includes:

  • SoapUI Pro (functional testing)
  • LoadUI Pro (performance testing)
  • ServiceV Pro (service virtualization)
  • Metrics & Data In One Place.
  • Create and maintain all the information relating to projects including test cases, data sources, and object components in one place for re-usability and management.
  • The entire ReadyAPI suite uses a central, standardized reporting function to support the measuring and displaying of data and metrics in real time.
  • Continuous Integration with ReadyAPI.
  • ReadyAPI provides many options for scripting to create functional, load, or security tests, using either Groovy or JavaScript.
  • With API Discovery, ReadyAPI will listen to live traffic of an API, recording responses and different interactions captured during the API activity.

#14 RestSharp

  • RestSharp is a popular HTTP client library for .NET.
    Featuring automatic serialization and deserialization, request and response type detection, variety of authentications.
  • JSON, XML and custom serialization and deserialization.
    Basic, OAuth 1, OAuth 2, JWT, NTLM are supported.
    HTTP call can be customized.
  • Send objects as the request body in JSON or XML, or as a form.

#15 SOAPSonar

  • SOAPSonar provides automated testing of each API independent of client, device or Consumer.
  • Automatically generating a UI to provide immediate feedback to developers, testers.
  • Validates each API independently, shortens the critical path and supports Agile development.
  • Fully automated, requires no scripting or maintenance thus reducing time and skill-set required.
  • Integration with variety test management software hence provides detailed progress reports.
  • Regression testing can be automated, providing early identification of changes to the API or versions.
  • Support for message and protocol layer identity standards allows for cloud testing.

#16 SOAtest

  • Broadest support for over 100 message formats and protocols.
  • With visual drag-and-drop tooling, users can create the most complex test scenarios without having to write a single line of code.
  • Test flow logic, complex assertions, looping, data driving, and keyword association, such as BDD with Cucumber, can all be easily built with minimal technical experience.
  • With its SMART API Test Generator plugin for Chrome, SOAtest monitors activity in your web UI from manual or exploratory testing and converts the behind-the-scenes API calls into meaningful API test scenarios.
  • Integration with existing CI/CD systems (i.e. Jenkins, TeamCity, Bamboo, Azure DevOps).
  • Collects REST and SOAP API coverage as part of test execution, reporting the test case coverage, missing or incomplete.

#17 TestMace

  • Creating and testing complex scenarios quickly.
  • Creating tests without actual programming.
  • Powerful autocomplete feature and highlighting of current variables values, functions etc.
  • A well-organized project structure and human-readable file format, which allow to store your project in the version control system repo and review tests and scenarios.
  • Advanced JS support in the built-in code editor, including the autocomplete feature and static analyzer.

#18 vRest

  • Record test cases by manually using your application once.
  • vrunner lets you execute your test cases just by running a command on your terminal.
  • Integration with Continuous Integration servers like Jenkins, TeamCity, Visual Studio Team Services etc. into vREST.
  • Automatically log issues into bug tracking systems like JIRA, Bugzilla, etc and notify team using Slack.
  • vREST lets you write test cases to validate your web/mobile application.
  • Supports Data driven testing. vREST lets you write test cases to validate your web/mobile application.
  • Schedule your test cases  using cronjob utility (in Mac or Linux) or Task Scheduler (in Windows).

#19 WebInject

  • WebInject is a free tool for automated testing of web applications and web services.
  • It can be used to test individual system components that have HTTP interfaces (JSP, ASP, CGI, PHP, AJAX, Servlets, HTML Forms, XML/SOAP Web Services, REST, etc).
  • Offers real-time results display and may also be used for monitoring system response times.
  • WebInject uses an XML API for defining and loading test cases. You can use WebInject without ever seeing its internal implementation.
  • Result reports are generated in HTML (for viewing) and XML 

#20 Paw

  • Paw is a full-featured HTTP client that lets you test and describe the APIs you build or consume.
  • It has a native macOS interface to compose requests, inspect server responses, generate client code and export API definitions.
  • Easily craft your HTTP requests, try and archive them.
  • Setup HTTP Headers, URL parameters, form-encoded POST key-values or text.
  • Get easy-to-read responses with well formatted headers and body.
  • Sync your projects with your team through Paw for Teams.
  • Simple request chaining (right-click in the UI, no code or globals).
  • Great workflow for when writing a fast-changing API

Conclusion

This list can go on and on however we have to stop somewhere. The above list is an effort to explore some different API testing tools which may not be the most popular ones but work well and perform effectively.

Though all API test tools listed above work well and are great options.however, there is no one-size-fit-all tool. There is no perfect tool for all organisations: Every organization has different requirements. 

So the key is – carefully considering the current and future project’s requirements and pros and cons of each solution.

]]> https://www.softwaretestingportal.com/api-testing-tools/feed/ 1 API Testing, Key Terminologies and more….. https://www.softwaretestingportal.com/api-testing/ Tue, 31 Mar 2020 23:27:34 +0000 http://www.softwaretestingportal.com/?p=929

An application programming interface (API) at its core is a formal specification that acts as a guaranteed contract between two separate pieces of software. We can also define API as an interface between two software applications by allowing them to communicate with each other.

APIs have become the focal point of programming improvement, interfacing and transferring information and rationale across frameworks and applications. Fortunately, testing them can enormously improve the productivity of your testing procedure overall, helping you convey programming quicker than at any other time. 

So What is API Testing!

Applications often have three layers: 

  • Data layer
  • Service (API) layer
  • Presentation (UI) layer
 

The API layer contains the business logic of an application – the rules of how users can interact with services, data, or functions of the app.  Since the API or service layer directly touches both the data layer and the presentation layer, it presents the sweet spot of continuous testing for Testing and Development teams. 

API testing involves testing the application programming interfaces (APIs) for functionality, security, Performance. Since APIs lack a GUI, API testing is performed at the Service layer.

Testing types in API Testing

1.Unit Testing: For testing the usefulness of individual activity. 

2. UI Testing: For testing the usefulness of UI as a component of start to finish coordination tests to guarantee the UI capacities true to form.

3. Functional Testing: Functional testing is simply a test of specific functions within the code base. These functions represent specific scenarios (successful cases) and error handling.

Due to the nature of the test, we should expect a few stated responses. We should expect either an error (and thus, the appropriate error codes and handling instructions) or a corrected response that bears the material we’ve requested.

Functional testing should be done with the positive scenarios as well as both errata and edge cases.

4. Load Testing: For testing the Functionality and execution under Load conditions. 

Load testing takes on a few different scenarios in order to ensure peak performance. The first of these scenarios is called the “baseline“, and tests the API against the theoretical regular traffic the API expects in normal, day-to-day usage.

A second load test is generally done with the theoretical maximum traffic. This is done to ensure that even during full load, methods are in place to safely throttle requests.

Finally, an overload test is performed to test the theoretical maximum and to add 10–20% additional traffic on the top. While this type of testing anticipates some sort of failure, it is more like a test of the API functionality to validate the error code generation and to ensure any failures are handled during high load.

5. Error Detection: For recognising any blunders, for example, special cases and asset spills. 

This type of test is focused on the actual running of the API. Other tests are concerned with the result of implementing the API in a scenario, error detection is concerned with the universal results of utilizing the API codebase. These types of tests have one of the following focuses:

• Monitoring: Compiled code is tested for various implementation errors, handler failures to ensure there is no security issue in the code base.

• Execution Errors: The code should respond to valid requests in a predictable, known way, and should fail invalid requests.

• Resource Leaks: Invalid requests, purposefully overflowing commands, and other “illegal but common” types of requests are submitted to the API to test for memory, resource, data, or operational leaks and insecurities.

• Error Detection: The code is put through known failure scenarios to ensure that errors are identified and handled.

6. Usability Testing : It is easy to hit the APIs on different platforms and test the usability of a software.

7. Security Testing: Security testing also includes penetration testing, and fuzz testing.

Security Testing includes validation of encryption methodologies and validating the design of the access control through user rights management and validating authorisation checks.

In Penetration testing, API is attacked by someone with limited working knowledge of the API to assess the threat vector from outside. These attacks can be limited to specific functions, resources, or processes, or can target the entirety of the API.

In Fuzz testing, massive amounts of random data, known as “noise” or “fuzz,” is forced into the system in order to attempt a forced crash, overflow, or any other negative behaviour. This is done to test the API at its absolute limits, and serves somewhat as a “worst case scenario.”

8. Interoperability and WS Compliance testing: This sort of testing applies to SOAP APIs and checks for following two aspects:

Interoperability between SOAP APIs is checked by ensuring conformance to the Web Services Interoperability profiles. By conforming to these guidelines and utilising these tests, interoperability between SOAP APIs can be confirmed

Secondly, WS-* compliance is tested to ensure standards such as WS-Addressing, WS-Discovery, WS-Federation, WS-Policy, WS-Security, and WS-Trust are properly implemented and utilized.

9. Discovery Testing : API testing helps in identifying APIs where the resources are not exposed.

Infiltration Testing: For identifying any vulnerabilities of an application from aggressors.

API Testing Best Practices

1. Deciding What to Test and what not i.e Create test cases for all possible API input combinations to get complete test coverage 

2. Use Realistic Data

3. Don’t Repeat Yourself.

4. Write clear tests that easily enable debugging.

5. Design the tests to run under different SUT configuration options.

6. Test Positive and Negative Outcomes

7. Use Data to Drive Dynamic Assertions

8. Track API Responses

9. Test the API for failures; until you get the output as failed.

10. Reuse test cases and monitor the API in production.

A simple API Testing process

Step 1: Create API testing requirements. To be able to plan API tests, we need to answer following questions:

  • We need to determine testing boundaries and requirements.
  • Purpose of the API and its target consumer;
  • Workflow of the application;
  • Features and functions of the API;
  • Other APIs this API interacts with;
  • Aspects of testing;
  • Problems we’re testing for;
  • Priorities in testing;
  • Desired output;
  • Definition of Pass and Fail.

Step 2: Establish the API test environment. After the functional scope, the next step is setting up an API test environment. It requires the configuration of the servers, databases, and every resource the API interacts with, depending on the project requirements.

Step 3: Make a trial API call. Before starting the testing, perform a Sanity test i.e. make a test API call just to check the API is operational.

Step 4: Define the input parameters. Plan all possible input combinations to verify the results to determine whether the API performs as expected.

Step 5: Create API test cases. After all the preparations are done, it’s the time to write and execute test cases followed by compare actual results with the expected ones. A good practice is to group them by test category. Examples of API test cases may include:

  • Positive and Negative test values to check the response is Passed or Failed.
  • When there is no return value in response.
  • Verification of response after data structure updates;
  • Validating resources modified by the API call.

Web Service / API Testing Terminologies

1. XML

  • XML stands for extensible Markup Language 
  • It’s a mark-up language much like HTML 
  • XML was designed to store and transport data 
  • XML was designed to be self-descriptive
  • It has sender information.
  • It has receiver information
  • It has a heading
  • It has a message body.

2. WSDL

  •       WSDL stands for Web Services Description Language
  •       WSDLs are an XML format that tells you how to access a web service.
  •       WSDL is used to describe web servicesWSDL is written in XML

3. SOAP

  • SOAP stands for Simple Object Access Protocol. It helps in exchanging structured information between computer networks.
  • SOAP allows communication between different operating systems using XML.

4. SOA

  • SOA (Service-oriented Architecture) is a way in which companies can organize software that can be quickly changed to respond to the requirements of the marketplace.
  • Web Services are small units of software that run in a network. They are typically written to handle a specific business process. Web services can be strung together in multiple ways and used by different applications to create the desired functionality.

5. REST

  • REST stands for Representational state transfer.
  • These are web services that provide interoperability between computer systems over the internet.
  • RESTful Web services provide a predefined set of stateless operations and allow requesting systems to access and manipulate textual representations of web resources.

6. SOAP vs REST

  • SOAP stands for Simple Object Access Protocol.
  • SOAP is a protocol. It defines some standards that should be followed strictly.
  • Web services following SOAP principles are called SOAP Web services
  • SOAP message request is processed slower as compared to REST.
  • SOAP supports only XML data format.
  • SOAP is not very easy to implement so it is preferred less.
  • SOAP requires more bandwidth and resources.
  • SOAP Message contains Envelope, body and header.
  • It does not use web caching mechanism.
  • SOAP is commonly used in payment gateways, financial and telecommunication services.
  • REST stands for Representational State Transfer.
  • REST is an architectural style.
  • Web services following REST Architectural style are called RESTful Web services
  • REST message request is processed faster as compared to SOAP.
  • REST supports data formats like plain text, XML, HTML, JSON, etc.
  • REST is easier to implement.
  • REST requires less bandwidth and resources.
  • REST Message is enclosed in HTTP and uses HTTP Method and URI (Uniform Resource Identifier)
  • It uses web caching mechanism i.e. data can be cached at the client.
  • REST is commonly used in social media, web chat and mobile services.

7. Web Services vs APIs

  • All web services are APIs.
  • It can only be hosted on IIS.
  • It is not open source but can be used by any client that understands XML.
  • It requires a SOAP protocol to receive and send data over the network, so it is not a light-weight architecture.
  • A Web service uses only three styles of use: SOAP, REST and XML-RPC for communication.
  • It only supports the HTTP protocol.
  • All APIs are not web services.
  • It can be hosted within an application or IIS.
  • It is open source and it can be used by any client that understands JSON or XML.
  • It is light-weight architecture and good for devices which have limited bandwidth, like mobile devices.
  • API may use any style of communication.
  • It supports the HTTP protocol: URL, Request/Response Headers, caching, versioning, content formats.

8. JSON

JSON stands for JavaScript Object Notation and was designed to be a lightweight data interchange format. JSON is definitely becoming more popular and is now replacing XML in certain situations for API data exchanging. The site www.json.org describes how JSON is built on two structures:
1. “A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed list, or associative array.
2. “An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.”

9. SOAP Message Elements

1) SOAP Envelope – The SOAP Envelope is always the top element in the message.

2) Header – the Header is optional and is the first child element to appear after the envelope. Headers can contain different types of application-specific information like security authentication or session management info.

3) Body – Sometimes referred to as the payload, the Body contains the actual message that shows the information for the recipient of the message.

10. Commonly used HTTP Methods

10.1 GET Method

  • The HTTP GET method is used to retrieve data. It is a read only call and data cannot be modified. If it is a successful request, without any errors, then a status of code of 200(OK) is received along with the data requested in the particular format which is generally JSON or XML.
  • GET method is considered a safe method to call as no corruption or modification of data will be done. 

10.2 POST Method

  • The HTTP POST method is a write method. It is mainly used to create a new resource by providing the input in Json or XML format.
  • When a POST request is made, a new resource is created with the values provided along with it. If it is an error free call, the status returned will be 201.
  • POST methods are responsible for modification of data and should be carefully handled.
  • These methods are not idempotent and thus can result in errors or duplicate records if called more than one time.

10.3 PUT Method

  • The HTTP PUT method is more like an update command. It is used to change the value of any resource whose original value was something else.
  • PUT methods can also be used to create resources but only if the the request is from the client and not the server.
  • PUT methods are also not safe as these are responsible for resource modification but if we call the same PUT request again then no changes are made i.e it is idempotent.

10.4 DELETE Method

  • The HTTP DELETE method, as the name suggests, is used to delete a resource.
  • If the request is made successfully then the status code returned is 200 (OK). These are also idempotent as if a resource is deleted once it cannot affect it.

11. HTTP Response codes

The HTTP response codes are divided broadly in five categories :
• 1xx – Informational codes
• 2xx – Success codes
• 3xx – Redirect codes
• 4xx – Client error codes
• 5xx – Server error codes

Some of the most common HTTP response codes used with REST are as follows :

  • 200 (OK): This code indicates that the request was made successfully.
  • 201 (Created): This response code indicates that request was successful and a resource was created. It is used to confirm success of a PUT or POST request.
  • 400 (Bad Request) : This code indicates that the data is in incorrect format.
  • 401 (Unauthorized) : This code indicates an authentication error.
  • 405 (Method Not Allowed) : This code indicates that HTTP method used is not supported for this resource.
  • 409 (Conflict) : This code indicates that there is a conflict request to create the same resource twice.
  • 404 (Not Found) :This code indicates that the required resource could not be found.
  • 500 (Internal Server Error) : This code indicates that there is some error on Server side.

The Benefits of API Testing

  • Early Testing –
    With API testing, once the logic is designed, tests can be built to validate the correctness in responses and data. We don’t have to wait for various teams to finish their work or for full applications to be built – test cases are isolated and ready to built immediately.
  • Easier Test Maintenance –
    UIs are constantly changing and moving around based on how they are accessed – browsers, devices, screen orientation, etc. This creates a nightmare scenario where tests are being constantly rewritten to keep up with the actual code in production. API changes are much more controlled and infrequent – often times API definitions files like OpenAPI Spec can help make refactoring tests only a seconds of work.
  • Faster Time To Resolution –
    When API tests fail, we know exactly where our system broke and where the defect can be found. This helps reduce time triaging bugs between builds, integrations, and even different team-members. The small, isolated footprint of an API test is perfect for faster MTTR stats, a valuable KPI for DevOps teams.
  • Speed and Coverage of Testing –
    100 UI tests may take 10 hours to run whereas 100 API tests could be run in 1 hour too. That means we can find and fix more bugs in less time.

Popular API Testing tools

Having the right process, tool and solution for API test are critical for success. Below are the tools which are quite popular in API testing space:

  • Postman
  • SOAP UI
  • REST-Assured
  • Swagger.io
  • JMeter
  • Katalon Studio

I’ll give a list of API Testing tools in next article where we ‘ll see some different API Testing tools.

]]>