How I Discovered Monkey Patching While Mocking AWS Cognito in an Angular App

I’ve been using AWS Cognito as the authentication piece to give users access to an Angular web project I’m working on.
Everything worked as advertised and I was happy with the result, until I started testing my authentication service.
That’s when things got interesting and I could not create a spy object on any of the Cognito classes.

Instead, I got…

1
2
3
4
5
6
7
8
9
10
AuthenticationService when .signInUser is working normally instantiate cognito AuthenticationDetails with correct user credentials FAILED
Error: <spyOn> : AuthenticationDetails is not declared writable or has no setter
Usage: spyOn(<object>, <methodName>)
at <Jasmine>
at UserContext.<anonymous> (http://localhost:9876/_karma_webpack_/src/shared/services/auth/authentication.service.spec.ts:33:40)
at ZoneDelegate.invoke (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-evergreen.js:359:1)
at ProxyZoneSpec.push../node_modules/zone.js/dist/zone-testing.js.ProxyZoneSpec.onInvoke (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-testing.js:308:1)
at ZoneDelegate.invoke (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-evergreen.js:358:1)
at Zone.run (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-evergreen.js:124:1)
at runInTestZone (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-testing.js:561:1)

Quick Answer

1
2
3
4
5
6
7
8
import * as AWSCognito from 'amazon-cognito-identity-js';

Object.defineProperty(AWSCognito, 'AuthenticationDetails', {
writable: true,
value: 'foo'
});

spyAuthenticationDetails = spyOn(AWSCognito, 'AuthenticationDetails');
  • This is strange because the same code in node.js throws TypeError: Cannot redefine property: AuthenticationDetails.
  • Angular uses zone.js to modify, or extend, all your async APIs.
  • We call this Monkey patching.
  • In my case, Angular Monkey patched amazon-cognito-identity-js and the result is we can execute Object.defineProperty without TypeError: Cannot redefine property: AuthenticationDetails.
  • Mocking AWS Cognito is simple, now that I know :)
  • Code examples on how to mock AWS Cognito in Angular.

What is Object.defineProperty

Object.defineProperty is a function which is natively present in the Javascript runtime environment and takes the following arguments:

1
Object.defineProperty(obj, prop, descriptor)

With Object.defineProperty, one defines a new property directly on an object or modifies an existing property on an object.

I found a very good StackOverflow answer, How to use javascript Object.defineProperty, and here are some points which helped me:

In Javascript, standard properties (data member with getter and setter) are defined by accessor descriptor.
Exclusively, you can use data descriptor (so you can’t use value and set on the same property):

  • accessor descriptor = get + set
    • get must be a function; its return value is used in reading the property; if not specified, the default is undefined, which behaves like a function that returns undefined.
    • set must be a function; its parameter is filled with Right Hand Side in assigning a value to property; if not specified, the default is undefined, which behaves like an empty function.
  • data descriptor = value + writable
    • value default undefined; if writable, configurable and enumerable (see below) are true, the property behaves like an ordinary data field.
    • writable - default false; if not true, the property is read only; attempt to write is ignored without error**

Both descriptors can have these members:

  • configurable - default false; if not true, the property can’t be deleted; attempt to delete is ignored without error**
  • enumerable - default false; if true, it will be iterated in for(var i in theObject); if false, it will not be iterated, but it is still accessible as public.

(** unless in strict mode - in that case javascript stops execution with TypeError unless it is caught in try-catch block)

(Side note: Also good to explore Object.preventExtensions(), Object.seal(), Object.freeze())

In my case, I actually marked the Cognito.AuthenticationDetails constructor function as overridable (writable).
But, you can only do that if configurable: true.

Here is the Mozilla example Object.defineProperty to explore:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
'use strict';
const object1 = {};

Object.defineProperty(object1, 'property1', {
value: 42,
writable: false
});

console.log(Object.getOwnPropertyDescriptor(object1, 'property1'));

// throws an error in strict mode
// Error: Cannot assign to read only property 'property1' of object '#<Object>'
object1.property1 = 77;

console.log(object1.property1);
// expected output: 42

If you want to explore my claim of read-only is the default, use console.log(Object.getOwnPropertyDescriptor(object1, 'property1'));.

Expected output:

1
Object { value: 42, writable: false, enumerable: false, configurable: false }

Let’s explore the configurable flag:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
'use strict';
const object1 = {};

// define a property
Object.defineProperty(object1, 'property1', {
value: 42,
});

// let's redefine the property
Object.defineProperty(object1, 'property1', {
value: 88,
});

console.log(object1.property1);
// expected output: 88

The example above will throw error:

1
Error: Cannot redefine property: property1

Resolve this error with configurable: true:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
'use strict';
const object1 = {};

Object.defineProperty(object1, 'property1', {
value: 42,
configurable: true
});

// let's redefine the property
Object.defineProperty(object1, 'property1', {
value: 88,
});

console.log(object1.property1);
// expected output: 88

Background

I have an authentication.service.ts and I want to test signInUser.
I reference amazon-cognito-identity-js for the Cognito functionality.

Here are a few tests I’ve identified to be implemented:

  • Cognito.AuthenticationDetails instantiated with correct credentials.
  • Cognito.CognitoUserPool instantiated with the correct UserPoolId and ClientId.
  • Cognito.CognitoUser instantiated with the correct ICognitoUserData object.
  • cognitoUser.authenticateUser was called only once.
  • On successful login, username must be returned.

(I’ve identified these tests by the comment: // test - ... in authentication.service.ts)

Mocking Cognito

My first mission: Test Cognito.AuthenticationDetails instantiated with correct credentials.

Because I reference amazon-cognito-identity-js, I thought mocks and spies would be the tools for testing AuthenticationService.signInUser.

The code below should do the trick…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
describe('AuthenticationService', () => {
let service: AuthenticationService;

beforeEach(() => {
// arrange
TestBed.configureTestingModule({});
service = TestBed.get(AuthenticationService);
});

it('should be created', () => {
expect(service).toBeTruthy();
});

describe('when .signInUser is working normally', () => {
let spyAuthenticationDetails: jasmine.Spy;
let result;

beforeEach(async () => {
//create spy
spyAuthenticationDetails = spyOn(AWSCognito, 'AuthenticationDetails');

// act
// system under test
service.signInUser({ userName: 'hello', password: 'password' }).subscribe(
value => {
result = value;
},
error => {
throw Error(error);
}
);
});

it('instantiate cognito AuthenticationDetails with correct user credentials', () => {
// assert
expect(spyAuthenticationDetails).toHaveBeenCalledTimes(1);
expect(spyAuthenticationDetails).toHaveBeenCalledWith({
Username: 'hello',
Password: 'password'
});
});
});
});

But failed with:

1
2
3
4
5
6
7
8
9
10
AuthenticationService when .signInUser is working normally instantiate cognito AuthenticationDetails with correct user credentials FAILED
Error: <spyOn> : AuthenticationDetails is not declared writable or has no setter
Usage: spyOn(<object>, <methodName>)
at <Jasmine>
at UserContext.<anonymous> (http://localhost:9876/_karma_webpack_/src/shared/services/auth/authentication.service.spec.ts:33:40)
at ZoneDelegate.invoke (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-evergreen.js:359:1)
at ProxyZoneSpec.push../node_modules/zone.js/dist/zone-testing.js.ProxyZoneSpec.onInvoke (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-testing.js:308:1)
at ZoneDelegate.invoke (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-evergreen.js:358:1)
at Zone.run (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-evergreen.js:124:1)
at runInTestZone (http://localhost:9876/_karma_webpack_/node_modules/zone.js/dist/zone-testing.js:561:1)

Based on the research here:

I’ve decided to go with the solution below:

1
2
3
4
Object.defineProperty(AWSCognito, 'AuthenticationDetails', {
writable: true,
value: 'foo'
});

I’ve updated my test and all was green…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
describe('AuthenticationService', () => {
let service: AuthenticationService;

beforeEach(() => {
// arrange
TestBed.configureTestingModule({});
service = TestBed.get(AuthenticationService);
});

it('should be created', () => {
expect(service).toBeTruthy();
});

describe('when .signInUser is working normally', () => {
let spyAuthenticationDetails: jasmine.Spy;
let result;

beforeEach(async () => {
// Confusion: Why does this output `configurable: true` in Angular world but not in Node???
// outputs: Object{get: function() { ... }, set: undefined, enumerable: true, configurable: true}
console.log(Object.getOwnPropertyDescriptor(AWSCognito, 'AuthenticationDetails')); configurable: true}

/********************* Magic *******************************/
Object.defineProperty(AWSCognito, 'AuthenticationDetails', {
writable: true,
value: 'foo'
});
/***********************************************************/

spyAuthenticationDetails = spyOn(AWSCognito, 'AuthenticationDetails');

// act
service.signInUser({ userName: 'hello', password: 'password' }).subscribe(
value => {
result = value;
},
error => {
throw Error(error);
}
);
});

it('instantiate cognito AuthenticationDetails with correct user credentials', () => {
// assert
expect(spyAuthenticationDetails).toHaveBeenCalledTimes(1);
expect(spyAuthenticationDetails).toHaveBeenCalledWith({
Username: 'hello',
Password: 'password'
});
});
});
});

Life was good and I continued working until the voice inside my head whispered:

“Why is this working…?”

Node behaves different to Angular

Although the chances of using a front-end library (amazon-cognito-identity-js) in a node.js server application is very low, I’ve created a quick test in node.js just to proof a point.

The test is following exactly what I’ve done in Angular, but without the extra package dependencies from Angular.

My investigation revealed Object.getOwnPropertyDescriptor(AWSCognito, 'AuthenticationDetails') returns configurable: false.

This is proof that AWS Cognito library is shipped with configurable: false.

Because it returns false, one would not be able to execute Object.defineProperty without an exception.
The AWS team does not want you to fiddle with their implementation of AWSCognito.AuthenticationDetails.

Below is my node.js test:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
const AWSCognito = require('amazon-cognito-identity-js');

describe('Investigaste Object.defineProperty of AWSCognito.AuthenticationDetails', () => {
it('It should throw "TypeError: Cannot redefine property: AuthenticationDetails" because Object.getOwnPropertyDescriptor returns configurable: false', () => {

console.log('Object.getOwnPropertyDescriptor(AWSCognito, "AuthenticationDetails"): ');
console.log(Object.getOwnPropertyDescriptor(AWSCognito, 'AuthenticationDetails'));

// this will throw exception because of 'configurable: false',
// meaning the AWS team does not want you to fiddle with this...and I agree, you shouldn't.
Object.defineProperty(AWSCognito, 'AuthenticationDetails', {
writable: true,
value: 'foo'
});

console.log(Object.getOwnPropertyDescriptor(AWSCognito, 'AuthenticationDetails'));
});

});

Output after running the test:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Randomized with seed 26638
Started
Object.getOwnPropertyDescriptor(AWSCognito, "AuthenticationDetails"):
{ get: [Function: get],
set: undefined,
enumerable: true,
configurable: false }
F

Failures:
1) Investigaste Object.defineProperty of AWSCognito.AuthenticationDetails It should throw "TypeError: Cannot redefine property: AuthenticationDetails" because Object.getOwnPropertyDescriptor returns configurable: false
Message:
TypeError: Cannot redefine property: AuthenticationDetails
Stack:
at <Jasmine>
at UserContext.it (/Users/ruanbeukes/repos/MockCognitoInAngular/node-property-descriptor-investigation/test.spec.js:11:12)
at <Jasmine>
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)

1 spec, 1 failure
Finished in 0.011 seconds
Randomized with seed 26638 (jasmine --random=true --seed=26638)

I’m confused…I’m sure in my Angular test, before the property modification code, I logged the exact same thing and the result was configurable: true.

How is that possible?!

Angular’s Zone.js and Monkey patching

Angular uses zone.js to handle change detection.
Because we have async code, Angular will Monkey patch all async APIs in order to managed the change detection.
(See resources below for a more in-depth look at zone'js as it does more than just change detection, it also helps with debugging)

A quick google revealed:

What is Monkey patching?
By definition, Monkey patching is basically extending or modifying the original API.
Now, zone.js re-defines all the async APIs like browser apis which includes set/clearTimeOut, set/clearInterval, alert, XHR apis etc.

amazon-cognito-identity-js will be re-defined by zone.js and also define the properties as configurable: true.
Because it sets configurable: true, it explains why I can define/modify object properties in the Angular world but not in the node.js world.

Below is the code that changes configurable: false to configurable: true from zone-evergreen.js.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
function rewriteDescriptor(obj, prop, desc) {
// issue-927, if the desc is frozen, don't try to change the desc
if (!Object.isFrozen(desc)) {

/******** Ruan: This is where Angular does something different compared to node.js **/
desc.configurable = true;
/************************************************************************************/
}
if (!desc.configurable) {
// issue-927, if the obj is frozen, don't try to set the desc to obj
if (!obj[unconfigurablesKey] && !Object.isFrozen(obj)) {
_defineProperty(obj, unconfigurablesKey, { writable: true, value: {} });
}
if (obj[unconfigurablesKey]) {
obj[unconfigurablesKey][prop] = true;
}
}
return desc;
}

Here is the stack trace when I’ve added a conditional breakpoint prop === 'AuthenticationDetails':

1
2
3
4
5
6
7
8
9
10
11
12
13
rewriteDescriptor	                                   @ zone-evergreen.js:2227
Object.defineProperty @ zone-evergreen.js:2178
__webpack_require__.d @ http://localhost:9876/_karma_webpack_/webpack/bootstrap:98
./src/shared/services/auth/authentication.service.ts @ authentication.service.spec.ts:201
__webpack_require__ @ http://localhost:9876/_karma_webpack_/webpack/bootstrap:79
./src/shared/services/auth/authentication.service.spec.ts @ environment.ts:13
__webpack_require__ @ http://localhost:9876/_karma_webpack_/webpack/bootstrap:79
webpackContext @ src sync \.spec\.ts$:9
./src/test.ts @ test.ts:20
__webpack_require__ @ http://localhost:9876/_karma_webpack_/webpack/bootstrap:79
checkDeferredModules @ http://localhost:9876/_karma_webpack_/webpack/bootstrap:45
(anonymous) @ http://localhost:9876/_karma_webpack_/webpack/bootstrap:152
(anonymous) @ http://localhost:9876/_karma_webpack_/webpack/bootstrap:152

The rewriteDescriptor function can be found in angular/packages/zone.js/lib/browser/define-property.ts in the Angular repository.

Conclusion

  • Be aware that Angular uses zone.js to modify, or extend, all your async APIs.
  • We call this Monkey patching and it is Angular’s way to deal with change detection.
  • If you build an Angular app and have unexpected side-effects in one of your async API libraries, start your investigation with zone.js.

Hopefully it saved you a few hours of research.

Use it…don’t use it :)

Code Samples

Mock Cognito In Angular

Resources

Create and Push Your Local Git Repository With Command Line Tool - Hub

What is hub (hub source)?

hub is an extension to command-line git that helps you do everyday GitHub tasks without ever leaving the terminal.

Why do I need hub?

It saved me the trouble to sign into Github, create the repository manually, just so I can clone it and work locally.

Current Workflow

Usually when I start a new Github repository, I’ll sign in and then create the new repository. The repository will then show me instructions on how to clone, HTTPS or SSH. Depending on my setup, I’ll clone and start coding.

My Use Case

I started a small throw-away-project, with a local git repository. A few hours later, I decided that the code might be useful and I wanted to formally push it to a remote source control server like Github.

My dream was to drop to the terminal and git -u origin master, that should create and push my local repository to my Github account…of course it is not that simple :)

I searched for a github cli tool and hub came up.
It runs on macOS, Linux, Windows and a few others.

After I installed it, these are the steps I followed to create and push my local repository:

  1. Open terminal and navigate to source code.
  2. Enter: hub create

hub requested my github username.
hub requested my github password.
hub requested my Two-factor authentication code.

After the hub create command, I had a repository in Github…magic :)

  1. git push -u origin master (yes, my dream line becomes a reality)

I also received a Github email notification: [GitHub] A personal access token has been added to your account

Conclusion

Although I only used the create command, I would encourage you to check out the hub manual for more interesting commands.

These two commands are the magic:

hub create
git push -u origin master

Use it…don’t use it :)

My "Reading" Celebrations 2019

September 2018, exactly a year ago, I got this crazy idea to read one book a month for the next year.
It is crazy because I’m not really your A+ student and I can always find something better to do, like sleeping, before I’ll read for fun :)
The only time you’ll find me reading is when I’m doing some research on a coding problem or when I’m learning some new tech to put into practice.

Basically, I read for educational purposes…not for fun :)

Which brings me to my next problem, how do one hack the system if you don’t like reading?

You turn to audio books.

I signed up for an Amazon Audible account and started “reading” away.
It’s a subscription model and your monthly fee provides you one book a month.

I vowed not to sacrifice my family’s time and added my own rule: Listen only on my daily commute to work.
That is, 30-35 minutes of learning pleasure a day, 5 days a week.

I set out to find anything to drive my career forward, topics like: people and success, body language, goal setting, negotiation skills and leadership.

I was very focused…first month, done.
Second month, done.
Third month, something odd is happening…

The longer my journey continued, the more I discovered all these things I thought would drive my career, actually helped me more in my personal life.
I realised all the extra time I put in was an investment into my family.
It helped me be a better husband, dad and just be a better person in general.

Books

Below is a list of the books I read.
I’m not going to give you a book review, you can read the review on Amazon :)
Instead, I’ll highlight parts which I found useful in my life.

  1. (PG) The Magic of Thinking Big - David Schwartz

This book addressed my self doubt and changed my thinking to: I can do it.
In fact, it pushed me to apply for a job I would never have considered before…and I got the job :)

  1. (A) Influence: The Psychology of Persuasion - Robert B. Cialdini

You can apply this book to all aspects of your life.
I applied some of the principles on my kids and was pleasantly surprised :)
Word of caution, there is a fine line between influence and manipulation.
Use your super power for good.

  1. (PG) Extreme Ownership - Jocko Willink, Leif Babin

Two US Marines sharing their leadership knowledge and how to apply the same principles in your business.
I later used some of those principles in my own life.

One bold statement from this book which might trigger interest:

There are no bad teams, only bad leaders.

You be the judge :)

  1. (A) Crucial Conversations - Kerry Patterson

Helped me on all aspects of my life! I would strongly advise to give this one a spin.
It helps with handling those uncomfortable conversations.
I found it very useful in everyday life.

  1. (PG) Maximum Achievement: Strategies and Skills That Will Unlock Your Hidden Powers to Succeed - Brian Tracy

Planning and setting goals.
Of course there are more than just those two points, but they are my key focus points.
A bit long but worked my way thought it.

  1. (G) The 10X Rule: The Only Difference Between Success and Failure - Grant Cardone

Good motivation to get you stirred up to do more than just what is needed.
Aim for the moon and you’ll hit the trees type of thing.

  1. (A) The Like Switch - Jack Schafer PhD, Ph.D. Marvin Karlins Ph.D.

It’s from an ex-FBI agent sharing years of experience on attracting and winning people over.
I liked this one very much, it helped me to be a better person and to put others first.

  1. (PG) What Every BODY Is Saying - Joe Navarro, Marvin Karlins

Also from an ex-FBI agent sharing how to speed-reading people.
I can now recognise small things my kids do before they turn into little demons :)

  1. (PG) The Charisma Myth - Olivia Fox Cabane

How anyone can master the art and science of personal magnetism.
This one helped me with a few ideas on public speaking.
It also pushed me to sign up for my first public speaking event…even though it was just 5 minutes :)

  1. (PG) A Whole New Mind - Daniel H. Pink

Shared some science behind how people think.
Also on how to spark the creative side of the mind.
Good book, enjoyed it!

  1. (A) Drive - Daniel H. Pienk

The surprising truth about what motivates us.
As always, Dan Pienk likes to use science to explain all his findings.
This helped me on all aspects of life, work and family life.

  1. (A) Never Split the Difference: Negotiating as if Your Life Depended on It - Chris Voss, Tahl Raz

An ex-FBI agent sharing some principles when he was a hostage negotiator.
Fantastic book and I apply the principles daily at work and at home.
It totally changed my point of view when you ask a question and the answer is “no”.
The word “no”, is the beginning of the negotiation.

It also helped negotiating my kids into bed :)

We all negotiate everyday, you don’t have to like it, you just have to accept it.

Rating Description
(A) Awesome and enjoyed it a lot
(PG) Pretty good
(G) Good

Celebrations

Today I celebrate the completion of my personal goal I set 12 months ago.
If I can do it, so can you!
It does not have to be books, identify your own interest and commit to it for a year.
You will surprise yourself.

Final thought

I believe life is about relationships, and I strive to treat anyone with respect regardless their social status or background.
These books just highlighted my own believes again.

Use it…don’t use it :)

Netlify - Add Google Analytics to Hexo on Production-Deploy but Not on Branch-Deploy

How would you know if your website is seen or used? Google Analytics.

For a while I’ve been using Google Analytics to track my blog.
It’s free and it gives me insight on which blog post others find interesting too.
But…I’ve been doing it wrong.

Tools and Workflow

My blog framework is Hexo and I have Continuous Deployment working with the help of Netlify.
When I commit code to master branch, my blog will automatically deploy for your viewing pleasure.
On feature branches, which represents a new post, I configured Netlify’s Branch-Deploy to make my draft post available via a special URL.
I like the Branch-Deploy feature, as I can quickly share it with a friend to get feedback but more importantly, I can compare how the webpage renders on different devices.

My Netlify Build & Deploy => Continuous Deployment => Deploy context settings:

Deploy all branches pushed to Netlify

Adding Google Analytics (GA) to Hexo

Assuming you have a Universal Analytics tracking ID (UA number), adding GA to Hexo blog is very simple.
(If you don’t have a UA number, see resources below to create one)

Just add your Universal Analytics (UA) number to your theme’s ../themes/[YourThemeName]/_config.yml file,
and also add a new ../themes/[YourThemeName]/layout/_partial/google_analytics.ejs file to Hexo.

Add Google Analytics to Hexo

Here is the content of the google_analytics.ejs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<% if (theme.google_analytics){ %>
<!-- Google Analytics -->
<script type="text/javascript">
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', '<%= theme.google_analytics %>', 'auto');
ga('send', 'pageview');

</script>
<!-- End Google Analytics -->
<% } %>

This works well and below is a sample of the Google Analytics code Hexo will generate in the head section of each page.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<head>
...
...
<!-- Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-7445566-1"></script>
<script>
window.dataLayer = window.dataLayer || [];

function gtag() {
dataLayer.push(arguments);
}
gtag('js', new Date());
gtag('config', 'UA-7445566-1');
</script>
<!-- Google Analytics End-->
</head>

My Mistake

After I’ve added GA to my blog, and I browsed the Branch-deploy URL (the test site), I realised GA recorded it as a real-time-active user.

I don’t want any stats recorded when I’m in Test/Draft, I’m only interested in Production stats…oops :)

My mission

How to get Hexo to generate Google Analytics (GA) on Production deployments only.

A quick hexo generate revealed when I remove UA number from the _config.yml, Hexo will remove all GA code from the pages too.

1
2
# Miscellaneous 
google_analytics:

I figured, if I can use Environment Variables to represent the Google Analytics UA tracking number, I would be able add the UA number to the _config.yml for Production deployments and remove it on Branch deployments.

My Solution

After a few minutes in the Netlify documentation, I discovered the awesome Stream Editor linux command sed.
This command will search and replace file content.

Now I have all the info to solve my issue.

Add GA_UA_PLACEHOLDER place holder text in the _config.yml which will be replaced on deployment.

1
2
# Miscellaneous 
google_analytics: GA_UA_PLACEHOLDER

Then, add the Environment Variables and sed (string replace) steps into the netlify.toml.

1
2
3
4
5
6
7
8
9
10
11
[build]
base = "blog"
publish = "blog/public"

[context.production]
environment = { GA_UA_PLACEHOLDER = "UA-7445566-1" }
command = "printenv && sed -i s/GA_UA_PLACEHOLDER/${GA_UA_PLACEHOLDER}/g ./themes/landscape/_config.yml && hexo generate && cp ../prod_headers.txt public/_headers --verbose"

[context.branch-deploy]
environment = { GA_UA_PLACEHOLDER = "" }
command = "printenv && sed -i s/GA_UA_PLACEHOLDER/${GA_UA_PLACEHOLDER}/g ./themes/landscape/_config.yml && hexo generate && cp ../branch_headers.txt public/_headers --verbose"

[context.production]: All steps under this section will execute when commit was detected on master branch.

environment = { GA_UA_PLACEHOLDER = "UA-7445566-1" }: Will set Environment Variable GA_UA_PLACEHOLDER to UA-7445566-1 when changes detected on master.

[context.branch-deploy]: All steps under this section will execute when commit was detected on any feature branch.

environment = { GA_UA_PLACEHOLDER = "" }: Will set Environment Variable GA_UA_PLACEHOLDER to empty string when changes detected on feature deployment.

printenv: Will list all the Environment Variables, just a debug thing I do :)

sed -i s/GA_UA_PLACEHOLDER/${GA_UA_PLACEHOLDER}/g ./themes/landscape/_config.yml: Will take _config.yml file as input and search for all occurrences of GA_UA_PLACEHOLDER and replace it with Environment Variable ${GA_UA_PLACEHOLDER}.

cp ../_headers.txt public/_headers: Will copy branch specific header file _header.txt to public folder with filename _headers, so that Netlify can apply the headers.

After these changes, the head section on each page for feature branches were GA free.
For master commits, the GA “magically” appeared.
Finally, my stats will reflect reality.

Use it…don’t use it :)

PS: Please feel free to leave a comment on how to improve this approach.

Google Analytics Resources

How to add a new website in Google Analytics
Google - Get started with Analytics

Approval Tests - Another Tool for My Software Test Belt

(Tech: C#)

Problem

I was working on a project where we refactored a particular class.
The class had a .Get() function, returning an Object.
Our mission was to implement a new class with the same .Get(), but the returning Object’s data would be retrieved differently.
This is a classic refactor, and thank goodness the team before us had sufficient unit tests :)

The unit tests were very easy to follow and with a few tweaks we had it implemented in no time.

Our next step was to implement integration tests.
This would definitely boost our confidence in our changes by comparing the old implementation to the new implementation’s result.

But how can I compare the objects without a lot of work from our team?

Plan of Attack

My plan was to serialize the objects to JSON string, and then compare the string results - see example below.
I would depend on the very popular Newtonsoft.Json to make it happen.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
// Don't think too much when you see the example,
// it's aimed to be very simple to get the point across :)

using Newtonsoft.Json;

// Xunit(pick your poison, MSTest, NUnit) integration test might look something like this:
public class AddressRepositoryTests
{
[Fact]
public void CompareOldAddressRepoWithNewAddressRepoTest()
{
// arrange
var oldAddressRepository = new OldAddressRepository();
var newAddressRepository = new OldAddressRepository();

// act
var oldAddresses = oldAddressRepository.Get();
var newAddresses = oldAddressRepository.Get();

// serialise
string oldAddressesJSON = JsonConvert.SerializeObject(oldAddresses, Formatting.Indented);
string newAddressesJSON = JsonConvert.SerializeObject(newAddresses, Formatting.Indented);

// assert
oldAddressesJSON.ShouldBe(newAddressesJSON);
}
}

This approach achieved the objective, and if I had automated integration tests running as part of my build pipeline, this would have been good enough.

In our case, we did not run the integration tests on every commit as we only needed a temporary test solution to prove our changes were working. At a certain point, when we were confident our code works, the plan was to remove the old implementation completely.

We also used Scientist.net to test in production…wait…WHAT!!!
Before you jump up and down, we are not testing in production :)
With Scientist.net, the old code will still executes but the new code will run as an experiment in production.
But that’s for another day :)

Back to my test scenario - because we ran our integration test on demand, I have found it difficult to track the difference in the output generated by the assertion tool, in my case Shouldly.

This was not a shortcoming of Shouldly, rather the Object returned was complex and the JSON string value returned was just to big.

My solution was to copy the oldAddressesJSON result into a file, and repeat the same thing with the newAddressesJSON so that I can compare the two files with a diff tool like P4Merge.
(Choose your Diff tools for Window or Mac - Kudos to Tobias Günther).

I found it easier to compare the result side-by-side via a diff tool as opposed to a text blob list from top to bottom in the generated output string. I preferred good old fashioned eyeballing the differences :)

Very soon I got bored with the copy-text-to-file and started thinking of a better way…

One of my team mates dropped Approval Tests, and I quickly spiked it.

Approval Test Approach

I found ApprovalTests.Net nuget package, compatible with: Nunit, MsTest, Xunit, MBUnit.

Package Manager install command: PM> Install-Package ApprovalTests.

It makes testing easier when one needs to eyeball the differences.
ApprovalTests.Net uses your default installed diff tool, in my case P4Merge, to show the differences visually.
I had it up and running very quickly just by looking at their examples.

The magic attribute is: [UseReporter(typeof(DiffReporter))].

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
using ApprovalTests; 
using ApprovalTests.Reporters;
using Data.Models;
using Newtonsoft.Json;
using System;
using System.Linq;
using Xunit;

namespace BuilderLibraryTests.ApprovalTests
{
public class SimpleApprovalTests
{
[Fact]
[UseReporter(typeof(DiffReporter))]
public void ThisIsASimpleTests()
{
var address = new Address()
{
PostCode = "6000",
StreetNumber = 12,
Suburb = "Huge Jackman :)" //aka Hugh Jackman
};

var jsonAddress = JsonConvert.SerializeObject(address);
Approvals.VerifyJson(jsonAddress); // Verify specific things, in our case JSON.

/*
//Another approach would be to use `Approvals.Verify(jsonAddress)`,
//which is just a string compare.
//The "disadvantage" with this approach is;
//one has to format your json result with 'Formatting.Indented'(see below).
//If you don't do it, you will get the object serialized in one line.
//Where with `Approvals.VerifyJson(jsonAddress);` you get the formatting for free.
//As always, use whatever gets the job done quickest.

var jsonAddress = JsonConvert.SerializeObject(address, Formatting.Indented);
Approvals.Verify(jsonAddress);
*/

// few other cool features I'd like to use when the scenario calls for it.
// Approvals.VerifyHtml(htmlString);
// Approvals.VerifyPdfFile(pdfFilePath);
// Approvals.VerifyXml(xmlString);
}
}
}

On First Run of Approval Tests

When you run the test the first time, the diff tool will pop up showing two file:

  • *.received.* - This is the actual result when your code was run.
  • *.approved.* - This is the expected result, the first time it will be empty as you still have to decide what is expected content.

First test run output result

You will also see two new files in your project.
In my example I verified json results Approvals.VerifyJson(jsonAddress), hence .approved.*json**.
It could be different depending on your Approvals.VerifyX method used.

Files created after running test.

The *.approved.* file is the one you want to add to your source control.
Think of it as your test’s answer sheet.
The content saved in that file will be used by the Approval Test library to verify the actual result.

Create Content For Approved File

How do we create the *.approved.* content?

Easy - you simply close your diff tool and it will prompt you to save the differences from the actual result into the*.approved.* file.

Files created after running test.

Now that you’ve saved the file, you have a valid *.approved.* file.

When you re-run your test, it will pass and the *.received.* file will automatically be removed by the Approval Tests library.

With a green test, we can commit the *.approved.* file with confidence to source control.

Test Workflow:
  • make changes to your code
  • update test, if required
  • run your test - let’s assume failed test
    • *.approved.* was created file
    • *.received.* was created file
  • diff tool will shout mismatch
  • save the change to the *.approved.* file
  • re-run your test
  • on green test, add/commit *.approved.* changes
  • on green test, *.received.* was automatically removed
  • new code is now covered

Suggestion: Configure your source control to ignore files with matching pattern *.received.*.
I’ve added *.received.* to my .gitignore file.

Approval Tests In Action

Quick sample of Approval Tests in action.

Conclusion

Approval tests are available in many flavours - C#, VB.Net, Java, PHP, Ruby or NodeJS.
A quick google will help you out.

I was very happy with the results Approval Tests brought to our project and would definitely recommend it for consideration in your next eyeball scenario.

Use it…don’t use it :)

Resources to broaden your knowledge

Apply Builder Pattern to Unit Tests

(Tech: C#)

Problem

I was working on a team (Team-B) where we were tasked with writing tests for a new implementation of the codebase.
The old implementation was fully covered, and our mission was to mimic the old tests in the new ones.

Sounds fairly simple, butttt…

The codebase is owned by another team (Team-A), and they have not contributed to the new implementation.

*How do we create unit tests which will speak to Team-A when they read the new tests from the imposters? :) *

Use the Builder Pattern

Here’s wikipedia’s definition on the builder pattern:

The intent of the Builder design pattern is to separate the construction of a complex object from its representation.
By doing so the same construction process can create different representations.

Because Team-A doesn’t have the knowledge of creating valid data objects for the new implementation, we used the builder pattern and created methods to abstract away the complex data structure setup.

Think of it from this point, if someone else wants to add more tests they should not have to dig deep into business logic on how to create a valid object. There should be a method they can call, and they will get a valid object every time.

One advantage I found was; the longer we worked on the test suite, the faster we got. This was because we started re-using builders created by other team members.

When I wanted to get a complex object in a valid state before I focus on my test, I’ll simply “new up” a builder and look for some interesting method on it. If not found, I knew it was a new scenario and method to be created.

Let’s look at some samples.

Samples

Disclaimer: These are all fictional - apply the principle and don’t get too bogged down in the example :)

(Samples github repo)

Let’s say another team is tasked with creating an Employee validator.
This team has no idea on how to create a valid employee.
Luckily another awesome team has created EmployeeBuilder, which they can re-use and focus on the logic of testing the validator.

Here is a requirement: An employee is valid when any address has an Australian postcode

Let’s dream up some test code for the employee validator…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[Fact]
public void EmployeeValidatorTests_EmployeeAddressShouldBeVALIDWhenAnyPostCodeFromAustralia()
{
// arrange
var builder = new EmployeeBuilder();

// no need to worry on how an Australian employee is created.
// imagine a very complex object here, with a lot of setup under the "With" method.
var employee = builder.WithEmployeeFromAustralia()
.Build();

// act
//system under test
var sut = new EmployeeValidator(employee);

// assert
// team focus on testing validator logic,
// not spending time figuring out how to create a complex employee object.
sut.IsValidAustralianAddress().ShouldBeTrue();
}

Let’s check out the EmployeeBuilder class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
public class EmployeeBuilder : BuilderBase<Employee, EmployeeBuilder>
{
AddressBuilder _addressBuilder = new AddressBuilder();

public EmployeeBuilder WithEmployeeFromAustralia()
{
_concreteObject = new Employee()
{
Name = "Bruce",
LastName = "Ozzy",

Addresses = new List<Address>
{
// default values
_addressBuilder.WithAustralianAddress().Build(),
}
};

return this;
}
}

public class BuilderBase<TBuildResult, TBuilder> : IBuilder<TBuildResult, TBuilder>
where TBuildResult : class, new()
where TBuilder : class, IBuilder
{
protected TBuildResult _concreteObject = new TBuildResult();

public TBuildResult Build()
{
return _concreteObject;
}

public TBuilder With(Action<TBuildResult> setAction)
{
setAction?.Invoke(_concreteObject);
return this as TBuilder;
}

public TBuilder With<TRequestBuilder>(Action<TBuildResult, TRequestBuilder> setAction) where TRequestBuilder: class, IBuilder, new()
{
setAction?.Invoke(_concreteObject, new TRequestBuilder());
return this as TBuilder;
}
}

public interface IBuilder { /* maker to indicate a builder object */ }

public interface IBuilder<TBuildResult, TBuilder> : IBuilder
where TBuildResult : class, new()
where TBuilder : class, IBuilder
{
TBuildResult Build();

/// <summary>
/// A generic way to set properties
/// </summary>
TBuilder With(Action<TBuildResult> setAction);

TBuilder With<TRequestBuilder>(Action<TBuildResult, TRequestBuilder> setAction) where TRequestBuilder : class, IBuilder, new();
}

You’ll notice I have TBuilder With(Action<TBuildResult> setAction) on the BuilderBase class.

It allows me code like this:

1
2
3
4
var builder = new EmployeeBuilder();

var actual = builder.With(x => x.Name = "Samurai Jack")
.Build();

I like it because one can easily see from the test what the intent is.
Abstracting this away into a method will hide the fact that the Name property changed.

There is also
TBuilder With<TRequestBuilder>(Action<TBuildResult, TRequestBuilder> setAction) where TRequestBuilder : class, IBuilder, new()
on the BuilderBase class.

It allows me code like this:

1
2
3
4
5
6
7
8
var builder = new EmployeeBuilder();
var employee = builder
.WithEmployeeFromAustralia()
.With<AddressBuilder>((e, addressBuilder) => e.Addresses.Add(addressBuilder
.WithSouthAfricanAddress()
.Build())
)
.Build();

My suggestion would be to have all builder classes only in your test project.
No need to have builders in production code as the data will come from the real source.

More samples can be found on my github repo.

(Btw, I’m using Shouldly for my assertions)

TestStack.Dossier - Check it out!

This is more in-depth “framework” and something I will definitely use on bigger projects.
If I needed more complex logic in my current implementation of the BuilderBase, I’ll upgrade to Dossier no questions asked.

TestStack.Dossier provides you with the code infrastructure to easily and quickly generate test fixture data for your automated tests in a terse, readable and maintainable way using the Test Data Builder, anonymous value and equivalence class patterns.

Conclusion

All the logic of creating complex objects is kept in one place, will help with future maintenance.
Writing tests in a fluent way makes the test more readable and also shows testing intent better.
Writing tests got faster as the builders library grew.
Cross functional teams don’t need the in-depth knowledge of a specific area, if there is a builder they can focus on their tasks.

I had fun using the builder pattern to create data objects in a fluent(ish) way for unit testing.
Will use it again in future.

Use it…don’t use it :)