How to verify the data accuracy of the EDI 834 file in Parasoft SOAtest?

 In this sprint, we have the new feature where we need to send outbound EDI 834 files to Washington Health Benefit Exchange (WAHBE).

The process is that BizTalk will create the canonical XML files that pass the validation error and then EDI 834 files will be generated automatically.

How to verify the data accuracy in EDI 834 line by line? It is really the challenge in SOAtest. Here is the way.

• We look for the canonical XML file name in the archive folder.
• From the Canonical XML file, we can get the EDI file name (Partial)

• Convert EDI file to XML format

• In XML Edit, this is the EDI 834 file BizTalk creates and SOAtest converts it to XML format .

• In XML diff, we parameterize the variables in the elements and then use it to do the data validation with step 4.

• We don’t need to validate every element in XML tag. We can put XPath in the Ingored Differences. 

From those steps above, it is really easy to validate the data accuracy of EDI 834  in SOAtest.

Parasoft SOAtest: Test execution in different environments via Test Configuration

In Parasoft SOAtest, if you have 20 .tst files where your environment reference points to QA environment, after developers push the code to UAT, do you still need to manually change them to UAT 20 times and then do the smoke tests?

Later, if you would like to do regression tests in QA environment again, do you still need to manually change them back 20 times?  

Here is one easy way to handle the scenario above using the Test Configuration in SOAtest.

• Right click the tst file
• Click Test Configuration
• Duplicate the Example Configuration and rename it
• Click Execution tab
• Enter the name of environment
• Close the window

If you want to run the tests in QA environment, just right click the tst files and choose Test Using->User-Defined->QA1. The tests will be run in QA1 perfectly.

The Parasoft SOAtest 9.9 upgrade means side-by-side installation with SOAtest 9.7

We have used SOAtest since last July. We were told we needed to upgrade SOAtest 9.7 to 9.9. After the installation, we found that it is not really the upgrade. It is called side-by-side installation. SOAtest 9.9 and SOAtest 9.7 are installed together in my laptop.

Here are 2 main issues after the “upgrade”:

• We need to reconfigure everything in Parasoft->Preferences.

• If we create new tests in SOAtest 9.9, we can’t use SOAtest 9.7 to expand the test scenarios in the individual .tst file. It is not backward compatible.

Why let users reconfigure the settings in SOAtest after the upgrade? It is not user-friendly. 

Fixed the issue of Parasoft SOAtest: XML assertor didn’t work

Today I noticed that in SOAtest,  the XML assertor of validating the elements in WCF response always passed even if the elements returned were wrong.

After looking into this interesting issue, I was not sure why another tester set up the following flow type: While pass/fail=1, which made the XML assertor passed all the time.

After changing the flow type to none, the test failed. 

Parasoft SOATest issue: Assertion tests still pass when the data returned is all blank in WCF response

Today I find one interesting issue in SOATest tool. Our developers have recently updated the namespaces in the data contracts that support “http”. For example,

Old: <ns1:Addresses xmlns:ns1=www.XXX.Com/member/06/01/2015

New: <ns1:Addresses xmlns:ns1="http://www.XXX.com/member/06/01/2015"

We have so many tests that it is not easy to tell which one has refreshed the WSDL in WCF methods. Our tests are using Value Assertion to validate the data elements in WCF response. After running all tests, all tests passed.

I noticed that in some tests, the data returned was all blank in WCF response. I could not believe that all assertion tests still passed.  That’s really surprising.

After updating WSDL again, the data returned can show all elements in WCF response. 

Fixe the issue in Parasoft SOAtest: Message: Could not find endpoint in client configuration file

In SOATest, we have the configuration file called Environments.env where we store environment variables. When running my BVTs today, some of them failed.

I notice that our team added a new path (/BusinessService ) in the URL. After updating it in Environments.env and running BVTs again, I saw the following error message. Closing and reopening SOAtest still could not solve the issue.

Could not find endpoint in client configuration file: WCF URL

After looking into this interesting issue, I notice “Constrain request to WSDL” in some tests was checked. After unchecking it and running BVTs again, all tests passed.

Parasoft SOAtest: Fixed the issue of line feed character ( )

Today one test failed in SOAtest due to feed character (&#10). In SOAtest, if I get customer data from SQL server , the data appears in Result As XML -> Edit and it shows the following data.

<LastName>Lin&#10</LastName>

&#10 is line feed character and it is not visible in SQL Server. I can use LEN(LAST_ NAME) to know how many characters in this field. 

How to avoid the issue of line feed character and carriage return using SQL query? After using the following one, the test passed in SOAtest. 

RTRIM(replace(replace(LAST_NAME,char(10),' '),char(13),' ')) LastName

Parasoft SOAtest 9.7 doesn’t support Common Table Expression (CTE) in SQL Server 2008 R2

If you have the following SQL query that uses Common Table Expression (CTE), you will see the error after trying to populate the data in SOAtest.

  

DECLARE @PolicyName varchar(20)='%Glucose%'

…

; WITH CTE as

(

  SELECT field1, field2

  from Policy PO  WITH(NOLOCK)

  WHERE field1 like @PolicyName

)

The following error shows up in XML data bank after running the test.

com.microsoft.sqlserver.jdbc.SQLServerException: Must declare the scalar variable "@PolicyName".

When I see this error, I feel so surprised because this SQL query is always working in SQL Server 2008 R2, but SOAtest complains about it using CTE.

How to solve this issue?  We need to declare this variable in SOAtest.

1. Double Click Scenario
2. Click Variables and add PolicyName like this

    3.Modify the following SQL query

; WITH CTE as

(

  SELECT field1, field2

  from Policy PO  WITH(NOLOCK)

  WHERE field1 like '${PolicyName}’

)

After running the test, the variable will be passed to SQL query in CTE and the test will pass without any issue.

Parasoft SOAtest: Fixed the issue of WCF: Content Type application/soap+xml; charset=utf-8 was not supported.

Today we had the new deployment to Test Environment. After the deployment, running tests via SOAtest showed the following error.

Message: Content Type application/soap+xml; charset=utf-8 was not supported by service http://xxxxxxxx.svc.  The client and service bindings may be mismatched.

Since we didn’t have any change in data contract in WCF, I needed to know what caused this issue. I tried the following ways.

Updated WSDL : the error still appeared.

Service Account: this one had the issue. After we fixed it, the error still appeared.

I  added one SOAP Client and added the URL. This time I got the different SOAP version. It just showed “Find” , not Find SOAP 1.2.  That explained why I saw the error message above.

Our Developer noticed that Bindings section was missing in web.config compared to the last build. After adding it and deploying to Test environment again, the error message above was gone.

Parasoft SOAtest: Change the default lock level to Check Out- Prevent other users from checking out and checking in

SOAtest doesn’t integrate with TFS very well and it has the merge issue. Only 1 person can check out one tst file. If 2 persons are working on the same tst file, once A checks in, B will lose his work. In this situation, when we check out the tst file,  we always select lock type- Check Out- Prevent other users from checking out and checking in.

However, the lock type of unchanged-Keep any existing lock is showing up by default. How can we change it to what we want? Here are the easy steps:  

Launch SOAtest-> Click Window->Preference->Team-Team Foundation Server->Source Control-> Choose Check Out- Prevent other users from checking out and checking in.

Next time if you would like to check out the tst file, it will show up by default .

SOAtest: Environments setting is below Test (.tst) [bad design]

In Parasoft SOAtest , if I create 1 Test (.tst) file , Environment is added below the .tst file automatically. I can switch to another environment (DEV, TEST, UAT..)  manually. What if I have 10 .tst files? It means that I have 10 environments.

If our developers change the data contracts in some WCF methods and I need to do regression tests in DEV environment first. In this situation, I need to manually change 10 times to point to DEV from TEST. After testing in DEV and we push a new build to TEST, I need to change 10 times back to TEST again.

Why not have a global environment where we can change it one time only?  I would say Environments setting below Test (.tst) is the bad design.

SOAtest, you should make your own product more user-friendly. 

Fixed the issue in SOAtest: Element could not be located or did not contain text content.

In SOAtest, I always select Value Assertion from XML Assertor to validate the data accuracy in WCF response format. Recently I have seen the following error message very often. For example,

Element "../PhoneNumberTen" could not be located or did not contain text content.

If I Click “Evaluate XPath” button, “No nodes found” message appears.

If I view the response data in the Traffic Viewer, PhoneNumberTen is blank. It means when the value is blank, SOAtest regards it as “No nodes found”.

Because the data is dynamic from SQL Server, it is possible to make the test failed due to the issue above. How do I fix this issue?

Add a new Conditional Assertion and use the following condition.

If Has Children Assertion = true

Then validate the value of PhoneNumberTen

It means when the data is blank, the validation will be ignored. 

After using this condition above, the tests pass!

Fixed the issue: Use Has Children Assertion to validate part of a string in SOAtest

Our Developer changed the error message in the WCF response format. Due to the long Error Message, how did I validate the error message in SOAtest? The best way in my mind was to contain some keywords such as System.NullReferenceException in SOAtest. 

Long Error Message

Text: System.NullReferenceException: Object reference not set to an instance of an object.   at

.................

System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)

Could SOAtest handle “String Contain”  like C#?  After looking into this interesting issue, I set up the following variable.

I tried “Has Content Assertion”, and it didn't work. I tried “Has Children Assertion" and it worked perfectly.  I would say it is nice to validate part of a string via Has Children Assertion in SOAtest.

Need to reconfigure all settings in SOAtest after TFS 2013 upgrade

Our IT department was doing TFS 2013 upgrade last weekend.  After TFS 2013 upgrade, our team saw some issues of SOAtest this week.

When opening SOAtest, I lost the connection to TFS and it showed the error message all the time.

Another team member could connect to TFS in SOAtest , but she could not get the latest version on the project level because “Get Latest version” or “Get Specific Version” was disabled. The interesting part was that she could still get the latest version in the individual tst file. But that was not helpful.

We need to fix this issue as soon as possible. Therefore, the best way to fix this issue is to create a workspace, import all test cases from TFS and reconfigure the settings. Here are the simple steps:

1)    Create a folder in C drive.

2)    Open SOAtest and use that folder as a workspace.

3)    Click file ->import and I can import all test cases from TFS.

4)    Click Parasoft->Preferences and reconfigure all required settings .

5)    Run BVTs and validate if all tests pass.

Another member mentioned that after TFS 2013 upgrade, if we used SOAtest to reconnect to TFS 2013, we should not be able to see the issues above. I totally agreed. 

Why bother testers to reconfigure all settings in SOAtest after TFS 2013 upgrade? 

Parasoft SOAtest issue: the characters after & are truncated in Traffic Viewer-Response- Element

This morning I noticed one test failed in SOAtest. After looking into this issue, I have the confidence to say this is the bug on SOAtest side.

We have the SQL query to get the test data and then pass the data in WCF method.

SQL query:

@Name VARCHAR (100)

….Where RTRIM (Name) like '%&%' (Initially we didn’t have this condition .I added it.)

Test Data:

It contains “&” . For example, Name is Ray FURNITURE & EQUIPMENT

SOAtest (DB tool)

Click Traffic Object -> Traffic Viewer-Response-Literal

Ray FURNITURE & EQUIPMENT

Click Traffic Object -> Traffic Viewer-Response-Tree

It doesn’t show the tree nodes. It only shows NewElement.

Click Traffic Object -> Traffic Viewer-Response- Element

results:

  resultSet:

    rows:

      row:

        Name: Ray FURNITURE

I used different data with & several times and I noticed that the result only showed the characters before &. I believe the characters behind & were truncated in SOAtest.

I used different data without & several times and the complete data appeared.

Next time if you see this & issue, don’t feel surprised in SOAtest.

WCF Test Date Range: Parasoft SOAtest doesn’t return the correct rows in Response Element

Today I found out an interesting issue in Parasoft SOAtest.

Scenario:  Input 2 date fields and one parameter to see if the rows returned from WCF method are correct.

Here is the basic SQL query to get the data where the condition is using 2 different dates. It returns 9 rows and also includes the MinDate.

Field1_name >=’MinDate’ and Field2_name <=’MaxDate’

Open WCF test client, do the same Scenario ,  invoke the WCF method and I get the same result: 9 rows.

What happens to Parasoft SOAtest 9.7?  In SOAtest, double click Traffic View-Traffic Viewer-> Click Request: I see 3 input parameters.

Click Response - >Click Element: I only see 7 rows which do not include the MinDate.

I have the confidence to say my configuration is correct in SOAtest. 

After modifying the following SQL query, all tests in SOAtests pass.

Field1_name >’MinDate’ and Field2_name <=’MaxDate’

Parasoft SOAtest : Fixed the error message- the maximum message size quota for incoming messages (65536) has been exceeded.

Our team is using Parasoft SOAtest to test WCF method. After executing all tests, the following error appears.

Message: The maximum message size quota for incoming messages (65536) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element.

In C# , I know it is easy to adjust MaxReceivedMessageSize in app.config and the issue will be going away.

But in Parasoft SOAtest, how to fix this issue? 

If we change the Default to Custom and uncheck Constrain request to WSDL, the issue is going away.

Lessons Learned in Parasoft SOAtest

I have used C# to do the test automation for many years. Currently I am working on the new project. The interest part is that the team uses Parasoft SOAtest to develop automation to test WCF methods. Here are some lessons I learned:

SQL query:

If we put SQL query in txt file, please don’t put any comment (- -) .  At the end of every line, please add more space. Otherwise, SQL query will not work.

Check out:

SOAtest doesn’t integrate with TFS very well. It has the merge issue. Only 1 person can check out one tst file. If 2 persons are working on the same tst file, once A checks in, B will lose his work. Therefore, one tst file is only for one WCF method.

Refresh Issue

If I duplicate set-up test and modify the data source column name in ‘Extraction”, sometimes this column doesn’t appear immediately in SOAP Client. Clicking “refresh” is not working either. I need to close SOATest and reopen it.

Global Variables:

Put the values in the Test Suite instead of putting them in local test scenario. If you put the values in test scenario and when executing all the tests, some variables will get confused and some tests will fail.

When I use this tool to develop test automation, I see so many issues in this product. My colleagues said Parasoft SOAtest costs 1 million dollars and adding one user needs to pay the license fee. 

It is not worth 1 million dollars because Parasoft SOAtest is not the high-quality product after using it.