WSO2 QA Test Framework - Fundamentals

WSO2 QA Test framework has been developed to replace repetitive manual test procedures followed during the release cycles. We identified the tests which provide much ROI with automation and used them for phase1 of the test framework development project.
With the introduction of WSO2 Carbon product platform in late 2008, all java based products are implemented using the base carbon platform. All products (9 all together) are released at the same day which is a very different experience specially for a QA team. All products are supposed to go through a set of common tests and a set of product specific tests. All are supposed to work on multiple application servers, multiple JVMs, multiple browsers, multiple operating systems, multiple DBMSs etc.. hell a lot of test combinations! The product count exceeded the number of people in test team. More products are expected to be introduced in near future. Therefore, the only viable solution to manage the QA process is to automating as more tests as possible.
So, we started implementing our automated test framework in 2009 March.
As I discussed in some previous posts, we chose selenium for automation. We used Selenium Remote Control Java client driver to drive selenium tests with Junit. However our tests were not restricted to web based selenium scripts. We have written numerous tests which used Axis2 ServiceClient API and some other API methods to invoke web services, sending messages via secure channels, reliable messaging, message mediation etc.
We derived our project structure which adheres to maven as given below. You can get a SVN checkout of 2.0.3 branch of the test framework (which is the most stable version at the moment) from https://wso2.org/repos/wso2/branches/commons/qa/web-test-framework/2.0.3

common
bps
registry
wsas
esb
gs
mashup
is
ds

Each project is built using its pom.xml at the root of the project directory. We used selenium maven plugin and surefire plugin to start selenium RC server and launch Junit tests respectively.
commons project is used to maintain tests which are used in all products. For example, org.wso2.carbon.web.test.common.RegistryBrowser is common for all products since all products have a common registry browser. We used svn:externals to link the common classes to relevant projects.
You can find all common classes at https://wso2.org/repos/wso2/branches/commons/qa/web-test-framework/2.0.3/commons/src/test/java/org/wso2/carbon/web/test/common/

In addition to that, commons project is used to store the test artifacts which are shared among multiple products. For example JDBC connector jars and keystores are used in all products and those are stored in commons/lib directory.

The other most important resource included in commons project is framework.properties configuration file. It is used to configure the test framework according to the test environment. Under the "global properties" section of the framework.properties file, you could find the following properties.

host.name=172.16.37.1
http.port=9763
https.port=9443
carbon.home=/home/charitha/products/wsas/wso2wsas-3.1.3
context.root=/wsas
browser.version=firefox
admin.username=admin
admin.password=admin
module.version=2.03

You must change these properties as per your test environment settings. http.port and https.port are the embedded tomcat servlet transport ports used in WSO2 Carbon products. (full explanation of these properties will be included in a future post)

Lets look at a product specific test suite. Go to wsas directory in your test framework checkout. This is the project used to run WSO2 WSAS specific tests. You will find the following files and directories at the root of this project.

lib
src
pom.xml
runAll.sh
runAll.rb
wsas_test_suites.txt

lib directory is used to store the test artifacts specific to WSAS such as axis2 services, jaxws services etc.
You can invoke tests using two different ways.

  • Direct maven invocation - you can run tests individually by passing a system property as follows
mvn clean install -Dtest.suite=usermanagement

  • Run tests through a shell script - You can run tests individually or as a whole suite or few tests at once using this way. Here, it is required to uncomment the selected tests in wsas_test_suites.txt. The runAll.sh shell script read the test names from wsas_test_suites.txt and invoke each test.
sh runAll.sh

In any of the above mechanisms, we call a central test suite class which takes care of calling the individual tests. For each project we have a separate AllTests.java class which extends junit.framework.TestSuite parent. In our example, org.wso2.carbon.web.test.wsas.AllTests.java is the TestSuite.

See https://wso2.org/repos/wso2/branches/commons/qa/web-test-framework/2.0.3/wsas/src/test/java/org/wso2/carbon/web/test/wsas/AllTests.java for more details about this class.

You will also notice in AllTests class that initBrowser() method of the BrowserInitializer class is called to launch the browser instance for a particular test as follows.

public synchronized static void initBrowser()throws Exception{


if (browser == null) {
browser = new DefaultSelenium("localhost", 4444, "*"+property.getProperty("browser.version"), "https://" + property.getProperty("host.name") + ":" + property.getProperty("https.port"));
browser.start();
browser.setSpeed("200");

}
}

You can easily try out WSO2 QA test framework once you download and start using any of WSO2 Carbon based product. This is not a detailed explanation of the all features available in our test framework. I will guide you through more information in future posts. If you encounter any issues while using the test framework, please drop us a mail - architecture@wso2.org

Stay tuned.. will post more on automation soon!

Comments

Post a Comment

Popular posts from this blog

WSO2 Stratos - Introducing WSO2 middleware Platform as a Service (PaaS)

Image
Yesterday we released our newest integrated cloud middleware platform, WSO2 Stratos . It is now online and ready for use! For the past few years, WSO2 has been doing many releases based on revolutionary Carbon platform , which helped to consistently improve the platform with introducing latest state-of-art technologies. The initial releases of Carbon product platform intended to improve the features of the individual products. For example, a lot of core features were introduced to WSO2 ESB, WSAS, Governance Registry etc.. Then we focused on ease of integration and more component based model. We introduced Equinox based provisioning model to build your SOA platform by picking and choosing the components as and when you need. Then, the core product platform has been enhanced to support multi-tenancy. With the multi-tenant supported architecture at hand, our brilliant development team was able to introduce the first ever comprehensive middleware PaaS. Why is this so important? Now, WSO2 C...
Read more

Common mistakes to avoid in WSO2 ESB - 1 - "org.apache.axis2.AxisFault: The system cannot infer the transport information from the URL"

In the next few weeks, you could expect a series of blog posts from me which explain the remedies to avoid some common mistakes which we do when working with WSO2 ESB . This is the first of many. During the initial rounds of testing your integration solution or even in production systems, you may have come across the following error. org.apache.axis2.AxisFault: The system cannot infer the transport information from the URL .... Some of you may have struggled hours figuring out a solution to address this issue. Let me explain this error in detail and possible causes of this. So, next time when you see this error in your WSO2 ESB setup, you will not have to spend time unnecessarily googling everywhere. There can be many different causes of this error. I have observed  four main reasons. Cause 1 This is the simplest cause. Suppose, you have a proxy service similar to the following. <proxy xmlns="http://ws.apache.org/ns/synapse" name="MistakesTest...
Read more

Working with HTTP multipart requests in soapUI

Image
You can use HTTP request test step in soapUI to submit messages with various Content-Types.  In this post, we will have a quick look into the multipart/form-data requests in soapUI. multipart/form-data requests usually come into action when you do HTML form submissions with file attachments. For example, have a look at the following HTML form post. <form action="http://localhost:8090/CKFileUploadApp/UploadServlet" method="post" enctype="multipart/form-data"> <input type="file" name="file"> <input type="text" name="paramName"> <input type="submit" name="Submit" value="Upload File"> </form> In the forms like above, we have file upload option as a form input where we can upload a file along with the form submission. In that case, the request's HTTP content type will be multipart/form-data. How can we simulate such a request using soapUI...
Read more