Automated build deployment

As a tester, the time we spend on each task is important, we need to keep identifying solutions that can save our time and effort. I will discuss one such solution we implemented during automated build deployment.

There are many tasks that we repeat several times, this will be exciting first few times when this task is new but once you get used to it, it becomes repeatative.

As part of my work I am responsible for deploying the client side builds to web server. There is a fixed process that needs to be followed always, this situation is appropriate for automation and the good thing is, it does not need much time and effort. First, I will highlight the manual process that is followed to achieve this task.

1.Checkout the code from svn with a specific revision number to folder in local machine

2.Run a shell script to minify the buid

3.Rename the minified build as per the naming convention

4.Deploy build to server using any of the ftp clients, we used winscp.

This process takes 15-20 mins for each build

If there is a mistake in build like

Incorrect revision no

Missing file

Incorrect svn url

This process needs to be repeated

The solution was simple, we used a shell script to automate this whole process. the only things we needed was svn commands, winscp commands & dos commands

This helped us reduce total deployment time from 15-20 mins to 5 mins

With the help of systems team we made this solution more optimized and brought down the deployment time from 5 mins to 1-2 mins

The systems guy introduced us to TeamCity which is a continous integration server, it has a lot of features that can help us achieve automated build deployment.

Here are the general steps for setting up the deployment taken from teamcity site, This can vary based on your requirement.

1. Write a build script that will perform the deployment task for the binary files available on the disk. (e.g. use Ant or MSBuild for this)
2. Create a build configuration in TeamCity that will execute the build script.
3. In this build configuration configure artifact dependency on a build configuration that produces binaries that need to be deployed
4. Configure one of the available triggers if you need the deployment to be triggered automatically (e.g. to deploy last successful of last pinned build), or use "Promote" action in the build that produced the binaries that need to be deployed. Consider using snapshot dependencies in addition to artifact ones and check Build Chains tab to get the overview of the builds.
5. If you need to parametrize the deployment (e.g. specify different target machines in different runs), pass parameters to the build script using custom build run dialog. Consider usingTyped Parameters to make the custom run dialog easier to use.
6. If the deploying build is triggered manually consider also adding commands in the build script to pin and tag the build being deployed (via sending a REST API request).
   You can also use a build number from the build that generated the artifact.

Build Script could be shell script or ant This build script will have the above manual process in form of commands, once the configuration is completed you can just click a "run" button from the dashboard, deployment is done automatically and the results displayed on dashboard. There is also facility to check the log file to debug if deployment fails.

You can also add auto sending of emails if a build is deployed successfully to the relevant user group. This can be added in script. 

There is also facility to pass parameters to build script

Selenium

This is my first post on selenium - A web based automation testing tool. I got in touch with selenium when I  was hired to automate an application and I choose to use selenium. I must admit that it was confusing for me at the start, but as I read and did more practice, I was better than before :)

I had used Watir before using selenium, It was natural for me to compare both. Advantage of selenium was that it had IDE unlike watir which made selenium look more easy. but today I feel there is no advantage because I don't use IDE at all. In real time projects most people don't often use IDE for automation.

For  a new tester who wants to learn automation, IDE is a good way for getting the feel of automation, motivating testers to do more, BUT there comes a point during the learning curve of a tester, where recording and running script is no more happy happy or motivating.

Next level requires some sort of scripting or programming skills, this is a challenge for some one who has never written a single piece of code. Either you give up and boast you know automation till record and playback or you start the challenging journey to learn.

Selenium provides a decent list of options to choose as a supporting language, this is more confusing to someone who has never programmed, which language to choose?

Initially its confusing & struggle to kickstart, but persisting will help you overcome and add to your learning curve. There are different tools provided by selenium that you need to understand and choose the options that you want to use for your automation.

SeleniumRC, Web-Driver, Grid, IDE, Different language options

Selenium Groups is a good place to join, you can hear and ask, you get lots of valuble learning tips.

In addition, there are different tools used if you are developing automation as a fulltime project, it almost similar to any develpment project that needs versioning tools, build tools, development environment..etc

Will be talking more about selenium experiences in upcoming posts

Have a good time :)

Is Playing a Game same as Game testing?(part2)

This is a continuation from my previous post

There are different categories of games, I will categorize them in only 2 ways

1.Play free
2.Play with real money

Play free, as you all know are available on many game sites and social networking sites, you just need to login and then play.

There are another category of games, gambling-oriented games that let people win or loose actual cash. Here, you create an account, deposit money and play....that's it.

Console games - Here you need to purchase hardware and the game software to play, you need to invest cash to buy

There are other ways of making a player invest real cash in a game in the form of buying virtual currency, gifts...etc

Now let me comeback to the topic of Game testing and Playing.

If you are a tester who is testing the first category of games that are for free or the second category for real cash, the approach of testing would be different. there would be different strategy because the aim of building the games is different in each case from a business perspective. The is taken into consideration while testing the games.

Now, If I am a game player or user who loves to play game, why the hell must I bother about the business perspective. My aim is entertainment. I want to play the game to enjoy. I will play the game and only complaint or get frustrated if I face some problem while playing, either I report the problem or don't comeback to this game or site. If I like the game I may give my feedback and suggest for more features.

A game player plays for himself to enjoy while a game tester tests the game from different stakeholders perspectives and sees that game adds value to both end-user and company.

In addition, a game tester also has to be technical to understand the the system which provide different features apart from the game itself. For example

1.If you are testing a single player client based game developed using client side technologies like javascript, html5,css, major testing efforts are focused on the game functionality, user experience and compatibility

2.If you are testing a multi player game eg; gambling games, which has client-server and database and a whole group of supporting servers, the testing scope and efforts increase. This kind of system is complex and a challenge for testers. Integration testing, database testing, system testing, backend testing, performance testing need to be considered in addition to the client side testing.

A tester needs to understand the whole system and should be able to create good tests that can uncover defects.Test team needs to be in sync with other teams in order to rollout a good game. The competition in the gaming market demands new features quickly to retain users and attract new one's. This puts pressure on  all teams including QA to rollout quality games ASAP.

The severity or importance of a missed critical defect in a game is felt when

1. your game crashes at peak user traffic and your users are seen on a competitor's site :)
2. player looses real cash eventhough he has won the game
3. game is not playable on a popular device
4. player's game is reset to level1 after he has crossed 10 levels spending hours on clicking next level

If you still feel that playing a game is same as testing a game you are kidding me? :)

I am not aware of how the console based games are tested or how the beta testers work. I would be interested in knowing that.

Is Playing a game same as Game testing?(part1)

Game testing is not what most people assume it to be. Yes, I said ASSUME. It's pity that people judge with whatever limited knowledge they have. I feel sorry for them, because every one has freedom to form their own opinions.

Few examples

1.I play free games on my mobile to pass my time

2.We are a group of friends who play for fun during breaks

3.I play to test my gaming skills

4.I earn money by playing gambling games

5.I am a professional online gamer

What do you call all these people in the above examples

Gamers?

Users?

Game testers?

Isn't there a difference in playing a game and testing a game?

Yes there is, this is what interests me and I will elaborate on this interesting aspect.

It is very natural to assume that playing is testing and testing is playing, but I would like to point out while playing do you really care about anything other than winning the game or reaching next level?

Every game's intention is to attract a user and make them involved in the game so that user gets addicted and keeps coming back. and a user's skill improves as he plays more and more and achieves new levels.

Now how is a game tester different from a user who plays a game

A tester needs be in control before he has the task of testing a game. there could be a tendency to get involved in a game. BUT here is the difference between a game tester and a game user

A game tester wears different hats to test the game to find bugs, these hats can be

F-hat - Functionality

U-hat - Usability

C-hat - Compatibility

UI-hat - User interface(graphics, colors, effects, animations)

E-hat - Entertainment(enjoyment, challenging)

A game user wears only one hat

E-hat - Entertainment

The hats concept I have used from the famous author Edward de bono's classic - "Six Hats" for thinking skills, this helps us in testing different aspects of a game. 

When I say games, there are different categories of games and the user experience is also different. will continue this in next post...stay tuned

First post this year - Automation Buzz

Don't get desperate to build castle in one day.

The context is, testers doing manual testing suddenly realize that they need to learn automation, because one of his friends told that automation is mandatory in resume these days.

So, my dear friend, goes to a automation tools forum and asks a question.

"I am new to automation, please guide me how I can learn it quickly, send me code, sample examples, tutorials...blah...blah..blah".

On one hand it's good that my friend is quick to act, as the saying does "Better late than never". But still, he needs to calm down and think, think and think.

He has got a good start and now he needs to plan how he will achieve this goal, what are the methods to achieve this goal.

-Ask yourself why you need to learn automation tool
-Research on what automation tools are available.
-What technologies they support
-Does it support the application you test in your company
-Do you need any programming knowledge
-Ask friends who have experience in automation
-Install tool, follow tutorials, get hands dirty, read, do and reflect

I can keep writing more...

Automation is good if applied sanely, but it cannot be a quick fix for each and every testing problem.

I just realized, this is my first post of this year.

Happy New Year :)

Rip the Pessimism

I was interacting with a Junior team member and he was saying that, "There is no importance for QA". He was referring to the speech of head of the company. 

Suddenly I did flashback about the year end speech. I remembered one of the statement that stuck me hard "we have delivered lot of products this year, this was possible due to quick iterations".

I was wondering what does he mean by quick iterations? Now I relate this with the juniors complain that QA is not valued. Then I tried to explain him. How are quick iterations possible? you cannot just develop+fix+deliver. This is not an iteration. develop+test+fix, This is called an iteration. test part is completely Ignored, people think that products are delivered in quick iterations without any testing efforts?

Ignorance at its best.

Quality of build delivered depends on many factors

1.Experience of a developer

2.Expertise on the technology and platform for which the app is built

3.Knowledge about the domain

4.Attitude for delivering quality code that has minimal defects if they are expected to be delivered quickly

These factors are never considered or not spoken at all. They are all assumed that they are part of a quick delivery process.

The role of QA in delivering a product in quick iterations is significant

QA gives developers 

-Freedom to make mistakes

-Experiment with design

-Helps test their solutions quickly

-Provides valuble information

-Give continous feedback

but there is no acknowledgement, because there is a lack of awareness from management. The only thing business/management is interested is to roll out the product as quickly as possible. credit being given to creator but not even a small part to the person who supported to make the product better for release. This is the sad part but we as testers have to live with it. Infact, we can proudly appreciate ourselves because we contribute without any expectations. I draw a parallel with the nature, it gives us so much still we don't appreciate and take it for granted :)

This is what I had to tell my junior team member to motivate him. I could see a broad smile on his face with pessimism turning to optimism.

Performance Testing - An Experience

The best part our minds is that it always wants something new to do. I had an opportunity to do some performance testing which was something different from my daily routine.

Performance testing is considered to be part of non-functional testing group. The term itself suggests that in this type of technique, testers measure the performance application, server or the database. This process is similar to any other testing process, only difference is the requirements and goals. 

My approach was to keep it simple. Start with a plan. Since I did not have any specific requirements, I had  to define a baseline requirements based on conversation with server architect & database architect.

My first step was to define a baseline requirement which was my performance goal. Now few questions arise in my mind

Are these goals for the whole application? or for specific components? I made it clear again by discussion with all stakeholders that this was for the whole application. This helps in moving forward because you need to clear with what you are doing.

Next step was to choose a tool for performance testing. what tools must I use open-source or commercial? so, I started analyzing performance test tools that will fulfill my goals. since the project did not have a budget to afford a commercial tool at this stage of project, I had a very easy decision, I had to choose a open-source tool. After some analysis, I chose Jmeter for several reasons.

-open source & free

-easy to use

-has lot of features 

-most important it fulfills my goal

Do we have an environment for performance test? because if you do a performance test anywhere, then it will have consequences like results may not be correct, others may get affected who are using the same environment..etc.

I choose a environment that was under control so that it neither affects the results or any other persons.

We are entering the execution phase now. we have set up the environment and we need to do several runs to determine the results and information during runs.

All runs may not be successful but each run will give important information even if it fails.

After every run there were some steps needed in the environment setup to get the application back to what we call the initial or clean state.

For ever run a report was generated that captured all the details and it was presented to all stakeholders. The architect's did some tuning to see that we achieve the performance goals.

Finally, Performance goals for the sprint were achieved. During this experience I learned a lot of things that I like to share.

1.Baseline requirements are very important, they act as a goal for the performance test

2.Environment must be set up correctly. 

3.Identify factors that can affect the test results

4.Load test tool can simulate a limited no of users from a client machine. we need to know that number.

5.The tests run within an intranet will give different results. This may not be the case in real world scenario

6.Discuss with architect's based on information in test reports, what tuning they are doing for removing the bottlenecks, so that you can focus more in those areas

7.Jmeter has both GUI and NON-GUI interfaces. It makes a huge difference

8.While running tests you need to be observing the activity in server and db.Logs is one of the sources.

I am writing this article as my thoughts flow. It may not be as per best writing guidelines or rules, apologies for that. I will try to improve with each of my write ups.