Geeks With Blogs
Theo Moore Questions... Morphology? Longevity? Incept dates?

PLEASE NOTE: THESE WORDS ARE MY ONLY MY OPINION. SPARE ME THE EXERCISE OF DELETING YOUR COMMENTS AND LAUGHING AT YOU IF YOU FLAME ME OVER THEM, TROLLISH ONE!

***BEGIN EDIT***

There is a gentleman who has commented on this particular post who, it seems to me, wants to argue semantics. Upon checking his blog, I have determined that he has a particular axe to grind with regards to "automated testing" (a term, it would appear, he considers an oxymoron). As I am not particularly interested in a philosophical argument, I have chosen not to explicitly respond to his post. Out of respect for his opinion, I have of course left his comments in place so a reader may form their own opinion. I do, of course, appreciate his time for the post; I just don't feel like arguing over what I feel are nit-picking points. I am certain he would not agree, and since that is case, I feel like we should agree to disagree and have done.

I will answer one of his questions, though: I am indeed male...or I was last time I had my DNA analyzed. The results are available on request. :-)

***END EDIT***

I've been asked to test out TestComplete for our organization. We are a Hp tool (Quality Center/QTP) shop, but we've been unhappy with QTP lately. QTP is a good tool, but it needs a serious overhaul (IMO) in order to remain competitive. It's sort of like Windows back at the Win 95 stage; it is so ubiquitous that it can limp around and still have good market share.

As an aside, QTP basically does web testing well, but it is not so good (again, IMO) at handing non-web apps. It also requires a plug-in (read as: costs more) for each type of app you want to test. You want to test Delphi? Buy the plug-in. ActiveX? Buy the plug-in, etc. It also wants you do special builds to make some of the plug-ins work (Delphi, for example). If you are testing interfaces with a third-party app (as we are) getting the special build is out of the question so QTP isn't an ideal solution. Honestly, it's pretty darn expensive, too. I don't know how much we pay (ok, I do, but I can't say what it is), but it's a LOT for what we get. Support alone each year is in the tens of thousands and this is for only a couple of licenses. I can honestly go on and on about the weaknesses of QTP because I've used it a lot. It does have many strengths (ease of use for new users being a major one), of course and I don't mind working with it. Heck, I guess I better not given it's the tool I've used most.

But this post isn't really about QTP. I only mention some of the sticking points I have with QTP so I can contrast them against TC.

TestComplete, right out the box tests just about any darn thing you want. Period. I threw it at Delphi and it works great. It loves .Net and Java apps (Ah, reflection, what a wonderful thing!). The Web testing is a *little* less simple than QTP, but that's workable. It also works well with embedded ActiveX in webpages, something that QTP always hiccups on.

TC supports many languages rather than just VBScript (ala QTP). You like DelphiScript? Knock yerself out. It handles it great. C# Script? Get to it! All good. Obviously, the choice of language is sort of minor as most of them operate more or less the same (with different syntax), but it's cool that it works with them all. Obviously, your language of choice is independent of the language the AUT is written in (except in some odd cases, but I haven't tried this yet).

TC uses two modes forĀ  test object recognition: standard and "open". Standard is good for black box testing, where you simply interact with the interface. Open is nifty, because it lets you interact with the objects and data behind the interface. You can insert test conditions, check object status, intercept function calls (I think), all kinds of stuff.

TC will interface with MSBuild, Team System, run nUnit/jUnit scripts, and even do your nails. It comes with a HTTP load testing tool built-in (unlike QTP which has no such thing...you are stuck using LoadRunner, as separate purchase). I've not tried the load testing tool yet, but for the price, how bad can it be?

My only criticism of TC at this point is this: if you are new to automation, this is NOT the tool to learn on. I've said many times that if you want to do good, workable automated testing, you are best served with a dev background. TC makes the dev background almost a necessity. Much like Rational Functional Tester (IBM's premier functional testing tool...it ROCKS, btw), a dev background is a requirement not just something that would be nice.

I believe we are going to purchase this tool in the near future to use with our new product (it's a .Net 3rd party tool we are going to customize) and I am sure I'll have more to say as I use it. My thought is that it's even possible I'll pine away after my dear old QTP.....

Maybe I'll never have to see the infamous QTP "General Error" (The QTP equivalent of saying "I am broke and I won't/can't tell you why) ever again....

Now, time to get TC to do my nails.....

Posted on Wednesday, November 12, 2008 9:34 AM | Back to top


Comments on this post: TestComplete: A very brief review

# re: TestComplete: A very brief review
Requesting Gravatar...
Theo,

A good comparison of tools....It helps...

I am surprised that you mixed up the terms "test" and "automate". I am sure you understand the difference. If it is your general style of writing to use these terms interchangeably - then it is fine. But you would sound lots better if you use right terminology for right context.

For example :

"QTP basically does web testing well" could have been "QTP basically works well for web GUI well" (note QTP can never test - it can only capture and playback gui actions - that is not testing)

"You want to test Delphi" could have been "you want to automate the GUI of an application written in Delphi"

"The Web testing is a *little* less simple than QTP" could have been "QTP identifies and manipulates web GUI well"

Shrini Kulkarni
http://shrinik.blogspot.com
Left by Shrini Kulkarni on Nov 13, 2008 1:13 AM

# re: TestComplete: A very brief review
Requesting Gravatar...
First to Theo--Great Review Man! Sounds like TestComplete can jump through a lot more hoops than QTP can. Pity New Co. is beholden to HP/Mercury otherwise, I'd suggest we evaluate this app.

Second--to the comment posted above--
I've been in Software Testing for 10 years and there are things that you can do with QTP that qualify as "testing." It is capablye of so much more than simple record and playback and in my estimate (warning VIOLATION OF Commandment #5 of the 10 Commandments of Egoless Programming) reducing QTP's capabilities to just record & playback is selling the application short on so many levels.

I've used it to verify applications are storing data to a database correctly using ADO DB connection objects. I've used the DOM model of a webpage to dynamically determine objects on the page (with a little help from my friends of course) so that the tests were not relying on the Object Repository.

So, frankly--castigating Theo on his choice of language in an industry with no standards regarding what is a Test Case versus Test Script versus Test Scenario only shows your ignorance and is in no way suggestive of Theo's lack of experience in this arena. Which btw he has considerable knowledge.
Left by MES on Nov 14, 2008 8:20 AM

# re: TestComplete: A very brief review
Requesting Gravatar...
Dear MES,

Thanks for answering my query on be half of Theo. I would still love to hear from her (I think theo is female not a male).

>> there are things that you can do with QTP that qualify as "testing."

Can you name them? I am especially interested to know what is testing according to you. QTP is a tool, a software that can help a human tester in executing some tests and do some comparision. Testing in my opinion is a deep human intellectual investigative activity that happens to cognitive engagement of human with the environment of software. It is way beyond what a tool can do. It is possible that testing according to you may be different from my definition. But, equating automation to testing is something that is difficult for me accept with some probing around.

>>> It is capablye of so much more than simple record and playback

Yes it is ... I did not comment on record/playback aspect. QTP can do lots of things. None --- none of them can qualify to be testing in totality. QTP helps a human tester in testing. I agree it does more than simple R/P

>>> I've used it to verify applications are storing data to a database correctly using ADO DB connection objects. I've used the DOM model of a webpage to dynamically determine objects on the page (with a little help from my friends of course) so that the tests were not relying on the Object Repository.

You are right. QTP can do all these. But none of these can be called "testing" on their own. They are tasks or constituents of testing. If you call these as testing - I can only say that your definition of testing is very narrow and is nothing more than comparing expected result to actual result.

>>> So, frankly--castigating Theo on his choice of language in an industry with no standards regarding what is a Test Case versus Test Script versus Test Scenario only shows your ignorance

I may be ignorant -- but I am open to learn from you and improve my understanding of testing. I feel that what theo is calling as testing is actually is automation. I am sure she knows the difference.

By saying there is no standard in terminologies, what stops me from calling automation as fish and testing as elephant. Blogs and public communications are open for criticism and scrutiny. So, I posted my view points - let Theo clarify what is the meaning of testing according to her.

>>> is in no way suggestive of Theo's lack of experience in this arena. Which btw he has considerable knowledge.

My intention was never to question and challenge Theo's experience. As a tester I challange anything that is testable.

Even my this comment is open for challenge

Shrini Kulkarni
Left by Shrini K on Nov 20, 2008 8:12 AM

# re: TestComplete: A very brief review
Requesting Gravatar...
M. Kulkarni--to respect Theo's wishes not to clog his blog with our dialog..I have posted a response to your comments above at my own blog:
http://geekswithblogs.net/MaryanneSweat/archive/2008/11/20/can-automated-testing-be-called-testing.aspx

Feel free to pop over there and have a read.
[ and anyone else who's interested for that matter =) ]
Left by MES on Nov 20, 2008 9:41 PM

Your comment:
 (will show your gravatar)


Copyright © Theo Moore | Powered by: GeeksWithBlogs.net | Join free