Breaking the Rules, Pushing the Limits

Testers should break the rules....

I come across a lot of Developer In Test roles these days, its a new trend that a large number of companies are adopting as they fully embrace Agile and BDD practices, or at least try. I think its good and productive to embrace Agile, its something that works and as a tester far better than waterfall era of giving everything to test in a short space of time at the end of a project. 

I am concerned however in the trend of the DIT role and what its doing to the test horizon, what worries me is this now huge lean on automation, and companies losing sight of what testing is really about, now it seems automating your Cucumber feature files are more important than actually finding bugs, and in some companies I have come across, leaving functional testing to the BA's whilst your DIT's just write automation code or even worse, making the developers that actually wrote the code now functionally test it (recipe of an apocalyptic ELE for your project).

Testing is about finding bugs, about hacking and breaking, its about pushing the application you are testing to the limit, and as a tester to get the golden egg, is to make the application fail, or even better crash. 

Can you find bugs automating? well if you are automating correctly yes, if you go beyond the UI layer and start interfacing into the service layer and back end systems you can learn a great deal about a system and how it works, and the most useful automation suites I have come across and developed do this, automation goes beyond the Selenium/Webdriver tests that have come to gain huge popularity and notoriety like Kim Kardasian after her sex tape, you have to get past the surface and beneath the sheets (no pun intended and certainly no links included).

Automation is your safety net, and if done correctly as part of the testing process a great and important safety net to have, but you must do it properly, and if you want to learn to do it properly follow these blogs, for me these are true experts in the automation field:

1. Nat on Testing - If i am having trouble automating a feature, i come here first 9/10 hes already done it and blogged about it.
2. The Evil Tester - Mr Richardson has a great Selenium/Webdriver course on Udemy, if you want to learn how to write automation code in java properly, I strongly suggest you take it.

Im writing this because I recently got dropped onto a project that was about to go live, there had been a huge push on automating the application, and indeed a lot of time and money invested in this task. When I got given the application to test, in the first 10 minutes I found a major bug, its one of the first things I test for when I get given an web application.....XSS. What really struck me even more was how easy it was find, I didn't have to pollute some AJAX requests going on behind the scenes, that I only found by looking in the Firebug Net tab, nope all I had to do was put some encoded js into the url after the search input parameter, it was really that simple. 

It certainly showed me that the testers were far to busy automating than finding bugs, they lost their identity, they were not the super hero testers that they should of been, they became developers writing automation code, a waste of testing talent.

So by all means automate, but remember why you are there, and why you got into testing in the first place, push the application to the limit, and break the rules. Ill leave you with a great clip from an average movie, here the test pilot and his pilot buddy are set the task to test out a couple of drone aircraft that apparently are suppose to wipe the floor with their human counterparts, study what Hal Jordan does to beat them, and try and take this metaphor with you to work, albeit without the green spandex.

All Testers Should Have A Database

SQLite Manager and testing data files.

I recently was given a task to test some data files, created by one system, processed by another then sent back to the system of origin in a transformed state (data munged with a few extra columns and data, standard ETL stuff).

In my experience, when data leaves its system of origin it pretty much goes into the unknown, it passes a boundary into a foreign system, and sometimes a foreign culture. It may come from a company culture where testing is of the utmost importance and enter into a culture where testing in itself is a foreign concept, needless to say it will not get treated in the manner in which it is accustom too.

We must be careful of making too many assumptions, a 'csv' file that contains data means different things to different people and more importantly different systems.For example files created on a Windows server may not be encoded correctly for a UNIX machine, some developers insist that csv values are contained in "double quotes", others prefer to omit the same, and some systems just will not handle special characters, no matter how much encoding and escape characters you include....Any ways back to the task.

After being ask to undertake this testing task alarm bells rang,I  thought I needed to dust of my old awk skills, but then I remembered my data-warehouse days, and how easy it was to test data once it was actually in a database. I had a SQLite instance running on my machine from some RoR stuff that I was messing around with, and decided to use it to test the data files, after all it was just gathering (binary) dust on my Lenovo.

At this point I could of imported the file using the SQLite shell, but I wanted to search for a quicker way so I did some digging and found a cool Firefox plug-in called SQLite Manager.

In a nut shell, you install it, open it up in Firefox, point it to your csv files and hit import, field names (if you have a header line) and table name (based on file name) get created automatically, great for beginners and experts alike, we all like to save some time. Here is a quick guide on how to get it up and running, please note you need a running instance of SQLite before you can use this plugin, here are some very good install instructions (you dont need to install the rubygems, but you should use ruby...it increases your street cred).

Here is what my csv file looks like:

 1. Open up SQLite Manager, for the sake of this example Ill create a new database.

2. Once the database is created hit Database option on the top left, and go all the way down to 'Import' this will bring up the import wizard, you should be presented with some options, below I am importing a postcodes.csv and set the options to import the file, as you can see you can even import data via SQL and XML.

3.  Hit 'OK' and you should get an option to modify the import before it begins, as you can see SQLite Manager has already figured out the column names for you, here you can change the table attributes

4. After a while, depending how many records are in your file, the import will complete and you will be ready to start running your sql queries against the file, just hit the 'Execute SQL' tab and test away.

I hope this gives you an idea of how easy it is to set up your own database, and how useful it can be for testing, here ive given you an example in how you can use it to test data files. Maybe you are doing performance testing, you could perhaps use it to store results and compare metrics between runs and builds, and with the power of SQL you could get some really good reporting out of it.

I advise all testers (especially consultants) to keep your own database on your Laptop/PC, you never know when it will come in handy, and something like SQLite uses less resource than your browser! dont believe me...check your process status's!.