Tuesday, January 21, 2020

Steve Jobs

Here is some of the qoutes from the audio book about Steve Jobs by Walter Isaacson:

Goal should be to build great products and lasting company, rather than just make profit.
Good user experience can only be given by controlling end to end user experience.
It is good to have some taste and class. Engage more with artists.
Always hire A players as A player will work with A players only. If you hire B player, then eventually you will be surrounded by C players. 
 
 

Tuesday, October 29, 2019

Continuous Integration and Delivery pipeline using Bitbucket, AWS CodeBuild, AWS CodePipline and AWS CodeDeploy - Part 2

In the previous post we looked at how to configure AWS Codebuild for CI/CD pipeline.
In this post we will look at configuring AWS CodeDeploy.
First Let's see what is CodeDeploy.
According to Amazon Web Services website, 
AWS CodeDeploy is a fully managed deployment service that automates software deployments to compute services such as Amazon EC2, AWS Lambda, and your on-premises servers.
So now let's see how we can set up AWS CodeDeploy.
Once you login to your AWS account, go to AWS CodeDeploy and click on "Create Application".
You need to first provide Application name (such as "MyFirstDeployment") and choose your compute platform (EC2, AWS Lambda, Amazon ECS). Here we will choose EC2 Instance as our comput platform.



Then click "Create Application".

Once the application is created you must create a deployment group. You can have multiple deployment group per application. Deployment group allows to have different deployment settings per different environment such as Production, Staging,Development or different deployment settings per different types of application in save environments such as Frontend Web application vs backend microservices.
Click  on the "Create Deployment Group" button.
Here first provide the deployment group name and choose an existing service role which has required deployment permissions (such as EC2 access permission, cloudwatch log creation permission, S3 permission if you are going to download artifacts from S3).



Now, in the "Deployment Type" section choose how you want to deployment to happen. "In-place" means every time you deploy the application, previous version of the application will be removed and new one will be deployed. This means that your application will not be available during the duration of the deployment.
In the "Blue/Green" deployment, each revision of the application is deployed to a different instance(existing or new), which then brought online after the deployment. Existing instance keeps running during the course of deployment and taken offline after the deployment. This way your application is always available even during the deployment.



Here we will stick to In-Place deployment for now.
After this you need to provide the Tags to select the environment in which you want to deploy.



Next you need to provide Deployment Settings and if you use Load Balancer.



We can skip the Trigger and Alarm section and create the deployment group.

In next post we will look at how to setup the AWS Codepipeline and connect the AWS CodeBuild, which we setup previously and AWS CodeDeploy which we setup in this post.

Enjoy!!

Thursday, October 3, 2019

Continuous Integration and Delivery pipeline using Bitbucket, AWS CodeBuild, AWS CodePipline and AWS CodeDeploy - Part 1

In this post I am going to show you how to develop a continuous integration and continuous delivery pipeline using AWS CodeBuild, AWS CodePipline and AWS CodeDeploy. I will use bitbucket as our source repository. But any other repository such as Github or Gitlab also can be used. Ofcourse, certain steps may defer as AWS Code Pipeline can pull the code directly from Github, but not from bitbucket.

So here are the steps:

1. Configure AWS CodeBuild to build the code by directly pulling from BitBucket and upload the build artifacts in S3.
2. Configure AWS CodeDeploy to pull the build package from S3(configured in above step) and deploy the application.
3. Configure AWS CodePipeline to get the build artifacts from S3 and deploy them using the AWS CodeDeploy application configured in step 2.


This will be three part series, and in this part-1 we will see how to configure AWS CodeBuild.

Part 2: Configuring AWS Code Deploy

Step 1: Configure AWS CodeBuild

Login to your AWS console and go to AWS CodeBuild.
Most of the steps to create a CodeBuild project are self explainatory. So here I will mention here the critical steps that you may want to get right.
Under the Source section, choose BitBucket and select "Repository In My BitBucket Account.".
Now choose your "BitBucket Repository" in which you have your code to build.
Now under the "Primary source webhook Events", check the box named "Rebuild every time a code change is pushed to this repository". Additional options will be available where you can configure the events on which code will be built.
For example, you can select whether the code should be built on every push, or on every pull request created or every pull request updated.
As depicted in below screenshot, I have configured my build project to build on everey push on the dev branch, but not to build when any tag is created or updated.



 Now it is time to configure the Environment under which the code is built. Here you have to select the whether you want to use AWS provided Image ("AWS Managed docker images") or custom image ("custom docker image"). 
For most of the common programming language runtimes and environments(such as dotnet, php, nodejs, java, golang) AWS provides, so choose "Managed Image", and then choose the operating system. Here, I have selected Ubuntu, Standard and aws/codebuild/standard:2.0 as Operating System, Runtime and Image respectively.



Now, as with any AWS service, you have to select a service role so that CodeBuild can build you project and upload the artifacts to S3 or use any other AWS services required.




Under the "Additional Configuration" section you can select timeout and specify if your build requires certificates, connection to VPC and compute requirements. Also, you can specify the Environment Variables here. Environment variables are helpful if you want to include custom build step depending on the environment or naming the build artifact based on the environment.




Now, comes the most important step, which is BuildSpec. You can write the commands to build your project using the BuildSpec file. In this section you can choose if build commands are included in a file (name buildspec.yml) in your project, or you can specify the build commands directly in the editor provided by AWS console.




Below is sample of the buildspec.yml, which includes commands to build nodejs project and package it as zip file.


As you can see above, in the first few lines, I provide the runtime environment(here, nodejs, version 10), in the "runtime-version".

In the build section, I have provided commands to build the project. For this node project, after running npm install and npm run build, I am copying the "node_module" folder to the "dist" folder, which contains all other project files except the node_modules folder.
In the artifacts section, I have specified the name of the zip file which contains all the file I want to include in the package. The artifacts section also allows me to specify the files I want to include or exclude in the package. Here I am specifying all the content of the dist folder, which is the output of the build commands mentioned above. I have included appspec.yml and deployment scripts which will be usefull for deploying the application. We will them later in the section about AWS CodeDeploy.

Next comes the Artifacts section, where you can provide the details about the S3 location where your artifacts will be saved.




As shown above, you have to select the S3 bucket(which should be pre-existing) where you want to save the artifacts and the name of the artifact zip file. You will notice that the artifact zip file name matches with the name I specified in the artifact section of the buildspec.yml file.

The Path option is the folder name in the S3 bucket where your artifact will be saved. So in this case, the artifact will be save as "my-codebuild-artifact-1/dev/my-api-dev.zip".

Once this configuration is done, you can click the "Create Build Project" button and your CodeBuild project will be created. Now start the build and once it is finished, you will see your artifact is saved in the above mentioned path.


Enjoy!!



Monday, June 26, 2017

Bugs error and software quality

Some notes:
  • Bugs are experienced failures, failures comes from faults within the software.
  • Faults are introduced in software when some process is skipped during the SDL, such as code review
  • Software cannot be tested 100%.
  • Test automation just makes software testing faster, it does not improve software quality.
  • Test automation should be context driven and adaptive.
  • People and managers should put time and money to improve skills of the people involved in software development to improve software quality.
  • Early feedback from actual users is important.
  • Tools and technology does not improve quality, it is how way use them affects the quality.

Monday, November 21, 2016

How to generate Random numbers

If you want to generate a random number for some business logic you are implementing, what would you do?

You would use Random class if you use Java or C#. Most programming language has some library function or class to give you random number.

But suppose you need to produce random numbers by your own without using any library function what would you do?

There are many algorithms to use to produce random number and here I will demonstrate a very basic algorithm which uses the modulo operator (%) in C#. The goal is not to come up with a foolproof algorithm to generate random numbers, the goal is to just use simple math trick to understand how random numbers can be generated. If you really need to generate random numbers in your programs then you should use the inbuilt library functions provided by the language or framework you are using.

You know what is modulo (%) operator is, right? It gives you the remainder when you divide the left hand number by right hand number.

so doing 20 % 20 will give you 0. And 20 % 19 will give you 1 and 20 % 18 will give to 2 and so on.

20 % 20 = 0
20 % 19 = 1
20 % 18 = 2
20 % 17 = 3
20 % 16 = 4
20 % 15 = 5
20 % 14 = 6
20 % 13 = 7
20 % 12 = 8
20 % 11 = 9
20 % 10 = 0
20 % 9 = 2
20 % 8 = 4
20 % 7 = 6
20 % 6 = 2
20 % 5 = 0
20 % 4 = 0
20 % 3 = 2
20 % 2 = 0
20 % 1 = 0

You can see when you divide 20 by numbers from 1 to 20 you get number 0 to 9 as remainders. The trick is the larger the dividend, larger the range of number you get as remainders.

So let’s set dividend d to a some large number, for the purpose of this post I will choose 10000. This will give you the range of 0 to 4999 as remainders.

But as you can see, this method produces sequential numbers, not random numbers. Well, on every iteration you use the remainder to produce a new dividend and you can see that instead of sequential numbers you are getting the random numbers.

So, let’s use below equation to produce a new dividend on every iteration, which uses the current remainder as new dividend:

remainder = a * current remainder + b % divisor

Using the above equation, we get below result, when a = 100, b = 100 and divisor is set to 19:

2100 % 19 = 10
1100 % 19 = 17
1800 % 19 = 14
1500 % 19 = 18
1900 % 19 = 0
100 % 19 = 5
600 % 19 = 11
1200 % 19 = 3
400 % 19 = 1
200 % 19 = 10
1100 % 19 = 17
1800 % 19 = 14
1500 % 19 = 18
1900 % 19 = 0
100 % 19 = 5
600 % 19 = 11
1200 % 19 = 3
400 % 19 = 1
200 % 19 = 10

Notice that the above equation produces a random number on every step but it repeats after few iteration. This is because the chosen values of the a, b and divisor.

Setting the divisor to a really large value can give us random numbers which may not repeat to soon. Such as below:

100 % 12345 = 100
10100 % 12345 = 10100
1010100 % 12345 = 10155
1015600 % 12345 = 3310
331100 % 12345 = 10130
1013100 % 12345 = 810
81100 % 12345 = 7030
703100 % 12345 = 11780
1178100 % 12345 = 5325
532600 % 12345 = 1765
176600 % 12345 = 3770
377100 % 12345 = 6750
675100 % 12345 = 8470
847100 % 12345 = 7640
764100 % 12345 = 11055
1105600 % 12345 = 6895 
689600 % 12345 = 10625
1062600 % 12345 = 930
93100 % 12345 = 6685
668600 % 12345 = 1970

Here, a and b is set to 100 and divisor is set to 12345, while current remainder is initialized to 0 at the start. Note that setting divisor to 12345 did not produce repeated numbers in the first 20 iterations, but it can still produce repeated numbers after few hundred iterations. But you get the idea, right?

The equation a + b * currentRemainder  % divisor is called Linear Congruential Generator. You can read more about it here.

Thursday, September 18, 2014

Tackling the Arrow head Anti-Pattern

Frequently you will see following type of code in your code base.

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
if(result != 1)
{
    if(someOtherResult == 101)
    {
        if(anotherValue == 500)
        {
            // do something
        }
    }
    else
    {
        // do some other thing
    }
}
return;
Here the code forms a shape of an arrow-head, as below:
?
1
2
3
4
5
6
7
8
9
if
    if
        if
            if
                do something
            end
        end
    end
end
If you see, the main logic is deep down into the nested condition, it increases the cyclomatic complexity of the code.
A better version of the same code could be as below:
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
if(result == 1)
{
    return;
}
if(someOtherResult == 101 && anotherValue == 500)
{
    // do something
    return;
}
// do some other thing
return;
The above code does a number of things to flatten the code and make it better:
  1. Validations are performed first and it returns at the first opportunity.
  2. Multiple nested conditions are combine into one (with “&&”(logical And) operator. If there are multiple expressions forming one such condition, they can be moved to a separate method returning boolean. That method can then be use in the if condition as below:
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
if(IsValidResult(someOtherResult, anotherResult)
{
    // do something
    return;
}
bool IsValidResult(int someOtherResult, int anotherResult)
{
    if(someOtherResult == 101 && anotherValue == 500)
    {
        return true;
    }
    return false;
}