Spring Rest Endpoint Test With MockMVC + ApplicationEvent

Problem Addressed

In a service oriented architecture, known as SOA, we have several microservices that get regularly deployed. The  strategy and various needs of deployment of such services has already been solved by applications like kubernetes. From a quality perspective there needs to be a high level test after any service is deployed or any configuration change is made to that service. One legacy method is to have a bunch of manual testers validate that the service behavior is working as expected. A better alternative is to use make an endpoint that gets called in every service whenever a service receives load or a configuration update. This endpoint then runs a series of tests that will validate that the service is in a good condition. When it is done it reports back to the endpoint that was called the status. You can then consume this anyway you prefer. Log it, trigger alerts, and etc.

SmokeTestResult Class

This is pretty much a copy of DropWizard’s HealthCheck class, except we need this to be serializable so that Spring can send the object into JSON. Jackson needs a public constructor to do this because it determines the class structure from Java’s reflection.

package com.domo.bedrock.health;

import com.fasterxml.jackson.annotation.JsonIgnore;

 * Created by derekcarr on 4/5/16.
public abstract class SmokeTestCheck {

    protected String name;

    public abstract SmokeTestCheck.Result check() throws Exception;

    public SmokeTestCheck(String name) {
        this.name = name;

    public String getName() {
        return name;

    public static class Result {
        private static final Result HEALTHY = new Result(true, null, null);
        private static final int PRIME = 31;

         * Returns a healthy {@link Result} with no additional message.
         * @return a healthy {@link Result} with no additional message
        public static Result healthy() {
            return HEALTHY;

         * Returns a healthy {@link Result} with an additional message.
         * @param message an informative message
         * @return a healthy {@link Result} with an additional message
        public static Result healthy(String message) {
            return new Result(true, message, null);

         * Returns a healthy {@link Result} with a formatted message.
         * <p/>
         * Message formatting follows the same rules as {@link String#format(String, Object...)}.
         * @param message a message format
         * @param args    the arguments apply to the message format
         * @return a healthy {@link Result} with an additional message
         * @see String#format(String, Object...)
        public static Result healthy(String message, Object... args) {
            return healthy(String.format(message, args));

         * Returns an unhealthy {@link Result} with the given message.
         * @param message an informative message describing how the health check failed
         * @return an unhealthy {@link Result} with the given message
        public static Result unhealthy(String message) {
            return new Result(false, message, null);

         * Returns an unhealthy {@link Result} with a formatted message.
         * <p/>
         * Message formatting follows the same rules as {@link String#format(String, Object...)}.
         * @param message a message format
         * @param args    the arguments apply to the message format
         * @return an unhealthy {@link Result} with an additional message
         * @see String#format(String, Object...)
        public static Result unhealthy(String message, Object... args) {
            return unhealthy(String.format(message, args));

         * Returns an unhealthy {@link Result} with the given error.
         * @param error an exception thrown during the health check
         * @return an unhealthy {@link Result} with the given error
        public static Result unhealthy(Throwable error) {
            return new Result(false, error.getMessage(), error);

        private final boolean healthy;
        private final String message;
        private final Throwable error;

        public Result(boolean isHealthy, String message, Throwable error) {
            this.healthy = isHealthy;
            this.message = message;
            this.error = error;

         * Returns {@code true} if the result indicates the component is healthy; {@code false}
         * otherwise.
         * @return {@code true} if the result indicates the component is healthy
        public boolean isHealthy() {
            return healthy;

         * Returns any additional message for the result, or {@code null} if the result has no
         * message.
         * @return any additional message for the result, or {@code null}
        public String getMessage() {
            return message;

         * Returns any exception for the result, or {@code null} if the result has no exception.
         * @return any exception for the result, or {@code null}
        public Throwable getError() {
            return error;

        public boolean equals(Object o) {
            if (this == o) {
                return true;
            if (o == null || getClass() != o.getClass()) {
                return false;
            final Result result = (Result) o;
            return healthy == result.healthy &&
                    !(error != null ? !error.equals(result.error) : result.error != null) &&
                    !(message != null ? !message.equals(result.message) : result.message != null);

        public int hashCode() {
            int result = (healthy ? 1 : 0);
            result = PRIME * result + (message != null ? message.hashCode() : 0);
            result = PRIME * result + (error != null ? error.hashCode() : 0);
            return result;

        public String toString() {
            final StringBuilder builder = new StringBuilder("Result{isHealthy=");
            if (message != null) {
                builder.append(", message=").append(message);
            if (error != null) {
                builder.append(", error=").append(error);
            return builder.toString();

The Smoketest Event

package com.domo.bedrock.service.event;

import com.domo.bedrock.health.SmokeTestCheck;
import org.springframework.context.ApplicationEvent;

import java.util.HashMap;
import java.util.Map;

public class SmokeTestEvent extends ApplicationEvent {
    private Map<String, SmokeTestCheck.Result> map;

     * Create a new ApplicationEvent.
     * @param source the component that published the event (never {@code null})
    public SmokeTestEvent(Map<String, SmokeTestCheck.Result> source) {
        this.map = source;

    public Map<String, SmokeTestCheck.Result> getHealthCheckMap() {
        return new HashMap(this.map);

    public void addHealthCheckResult(String id, SmokeTestCheck.Result result){
      map.put(id, result);


The Endpoint each microservice will have

@RequestMapping(value = "/smoketest", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
public Map<String, SmokeTestCheck.Result> requestSmokeTestHealthCheck() {
    Map<String, SmokeTestCheck.Result> map = new HashMap<>();
    SmokeTestEvent smokeTestEvent = new SmokeTestEvent(map);
    return smokeTestEvent.getHealthCheckMap();

This will publish an event that the service will consume by implementing an application listener. When the listener is finished it will return all the results synchronously.

Default Application Listener

Simply extend this class and implement the method that returns the SmokeTest.Result information. This can run anything from simple ping/pong endpoint tests to a full behavioral test of the service.

package com.domo.bedrock.health;

import com.domo.bedrock.service.event.SmokeTestEvent;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;

import java.util.List;

 * Created by derekcarr on 4/4/16.
public abstract class SmokeTestListener implements ApplicationListener<SmokeTestEvent> {

    protected static final Logger LOGGER = LoggerFactory.getLogger(SmokeTestListener.class);

    protected List<SmokeTestCheck> tests;

    public abstract void onApplicationEvent(SmokeTestEvent event);


 Unit Test Endpoint With MockMVC

This has extra dependencies because of our deployment manager and preauthentication strategies, but this allows us to create a mock of the event and make sure that the endpoint returns what is expected. There is a better way to validate this – using json to do the validation, but we had conflicts with the various versions.

package com.domo.maestro.service;

import com.codahale.metrics.MetricRegistry;
import com.domo.bedrock.health.SmokeTestCheck;
import com.domo.bedrock.maestro.metrics.MaestroCustomMetricsAppender;
import com.domo.bedrock.service.CoreAutoConfiguration;
import com.domo.bedrock.service.ToeProvider;
import com.domo.bedrock.service.event.SmokeTestEvent;
import com.domo.bedrock.web.WebAutoConfiguration;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.web.HttpMessageConvertersAutoConfiguration;
import org.springframework.cloud.autoconfigure.RefreshAutoConfiguration;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationListener;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
import org.springframework.core.env.MutablePropertySources;
import org.springframework.core.env.PropertySource;
import org.springframework.mock.env.MockPropertySource;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.testng.AbstractTestNGSpringContextTests;
import org.springframework.test.context.web.WebAppConfiguration;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import org.springframework.web.context.WebApplicationContext;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

import java.util.EmptyStackException;

import static org.hamcrest.Matchers.containsString;
import static org.mockito.Mockito.mock;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultHandlers.print;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

 * Created by derekcarr on 3/31/16.
@ContextConfiguration(classes = {
public class MaestroResourceTest extends AbstractTestNGSpringContextTests {

    public static class resourceTestConfig {
        public MaestroResource maestroResource(ApplicationContext applicationContext1) {
            DefaultHandlerProvider defaultHandlerProvider = mock(DefaultHandlerProvider.class);
            MetricRegistry metricsRegistry = mock(MetricRegistry.class);
            ServiceContext serviceContext = mock(ServiceContext.class);
            MaestroCustomMetricsAppender maestroCustomMetricsAppender = mock(MaestroCustomMetricsAppender.class);
            return new MaestroResource(defaultHandlerProvider,metricsRegistry,serviceContext, maestroCustomMetricsAppender, applicationContext1);

        public SmokeEventListener smokeEventListener() {
            return new SmokeEventListener();

        public static PropertySourcesPlaceholderConfigurer placeHolderConfigurer() {
            PropertySourcesPlaceholderConfigurer pspc = new PropertySourcesPlaceholderConfigurer();
            MutablePropertySources propertySources = new MutablePropertySources();
            PropertySource ps = new MockPropertySource()
                    .withProperty("requireSecurityAdapters", "false")
                    .withProperty("authenticationKey", "hippopotomonstrosesquipedaliophobiahippopotomonstrosesquipedaliophobia")
            return pspc;

        public ToeProvider toeProvider() {
            return mock(ToeProvider.class);


    public static class SmokeEventListener implements ApplicationListener<SmokeTestEvent> {
        public void onApplicationEvent(SmokeTestEvent event) {
            event.addHealthCheckResult("Test all the awesome things!", SmokeTestCheck.Result.healthy());
            event.addHealthCheckResult("Something Terrible has happened", SmokeTestCheck.Result.unhealthy("There are no more cookies in the break room."));
            event.addHealthCheckResult("Throw your hands Up - they're playing Clint's Song", SmokeTestCheck.Result.unhealthy(new EmptyStackException()));

    private WebApplicationContext wac;
    private MockMvc mockMvc;

    private String expected = ",\"Something Terrible has happened\":{\"healthy\":false,\"message\":\"There are no more cookies in the break room.\"},\"Throw your hands Up - they're playing Clint's Song\":{\"healthy\":false,\"message\":null}}";

    public void setupSpec() {
        mockMvc = MockMvcBuilders

    /** test for {@link MaestroResource#requestSmokeTestHealthCheck()} */
    public void test_smoketest_endpoint() throws Exception {
                // HTTP 200 returned
                .andExpect(content().string(containsString("Test all the awesome things!\":{\"healthy\":true,\"message\":null}")))
                .andExpect(content().string(containsString("Something Terrible has happened\":{\"healthy\":false,\"message\":\"There are no more cookies in the break room.\"}")))
                .andExpect(content().string(containsString("Throw your hands Up - they're playing Clint's Song\":{\"healthy\":false,\"message\":null}")));



Finally Can Sleep on a Noisy Street


No Sleep MemeMy house is between a busy highway and a railroad. Living by a railroad doesn’t really bother me, but we have a street crossing really close to my house and by law railroads are required to blow their whistle for the international morse code “Q”,  which comes from the times when a queen would be on board a ship and they would blow “Q” to say they have the right of passage because of royalty onboard. Either way the whistle is loud, and having a highway right outside my bedroom window brings lots of traffic noise, trucks clinking and clashing, and random high frequency/pitch sounds. During the summer I just put a window fan in because the nights cool off enough I can blow cool air into my bedroom. The fan produces a great white noise, but during the winter I take the fan out of the window for obvious reasons . . . All of these random sounds wake me up in the middle of the night and cause me to toss and turn quite a bit. I’m so tired of not getting a restful night sleep. After hunting for a while I decided to buy a white noise machine.


I need a machine to make white noise for sleeping. There are several cheap ways to do this – anything from apps on the phone to having a bedroom phone making noise. I have tried phone apps and I don’t really like them because the speaker’s noise just doesn’t create a restful sleep – for whatever reason I am more awake when I use my phone for white noise. During the winter I really don’t want a fan because my wife will get too cold. The solution is to just get a white noise machine. I ended up buying the Sleep Easy listed below. I like this much better than a phone because a phone only has one speaker which means you only get one position in the wavelength of the sound being generated. Having the Sleep Easy produces sound in a circle location so the waves of sound propagating are  more fulfilled. I hear more positions of the waves making the white noise more relaxing. It just sounds deeper and richer.

After getting a white noise machine I have been able to sleep without interruptions better. My wife grinds her teeth at night and I even sleep through that and she can sleep better through my sleep talking. Occasionally I will also work from home and having a house full of people gets noisy quick. I will take the machine to my office and turn it on, which helps me focus better for work rather than hearing the television upstairs. I am going to buy another one for my newborn daughter because it helps her sleep even when there are noises going on around the house. It’s a nice filter. She loves to see what’s going on around her and this helps drown the excitement. The only complaint I have about these white noise machines is they don’t have battery power, which makes taking them for traveling harder as not everywhere my family goes has standard 120V power or power in general.

[AMAZONPRODUCTS asin=”B000KUHFGM”]This is the main brand machine for generating white noise. My has one of these and he loves it. Dohm white noise has been around for a long time and is well tested. [/AMAZONPRODUCTS]

[AMAZONPRODUCTS asin=”B00GFSF402″]The Sleep Easy is the off brand white noise machine that has all the same functionality as the main brand, but for half the cost. This is how I am able to buy one for my bedroom and my daughter’s bedroom. My dad has tried both the DOHM and Sleep Easy and says they are the same in his mind. He prefers this sleep easy because of the price also. He has bought two extra Sleep Easy white noise machines for other reasons just because of the good price.[/AMAZONPRODUCTS]

How To Build An API

how to build an api

What is an API

API stands for application program interface. In a nutshell an API is a way for external applications to access information and queries from your program. Twitter for example has a a REST API that allows you to access all types of data from twitter. REST API’s are among the most common types of APIs. A REST API allows you to make requests using standard http methods. You can make a GET request to get data, POST request to insert data, PUT request to update data, and DELETE request to remove data. Most companies give a limited number of free API calls into their applications. As your need for their APIs increases they start charging more for API calls. A call is counted each time you call an endpoint. When a company limits their endpoint requests you can be smart and request more data through a single endpoint query.

I have a good example of working around limited API calls. Salesforce is the worlds leading CRM and they have an open api you can use to access data, however, they severely limit the number of times you can call an endpoint in their system. I have worked for companies that interface for Salesforce and we needed to get Lead data from them. Instead of calling an http GET endpoint to get each lead individually we would query as many as we can and just store them on a local server. Another thing we did to limit our endpoint calls was to batch process them. We would store all the information we needed to access and then every 15 minutes or so make batch queries. This limited the number of queries we needed to call by about 70%.

Why an API

There are several reasons large applications should use an API as one of their layers of abstraction. An API is not the same thing as a plugin or an application, but the business models are similar. One reason the iPhone became so successful is because by the third generation apple introduced the app store. Salesforce generates revenue for so many people by allow apps or plugins to be installed on their application from third parties. An API is the same idea. You want to have people be able to access, share, and improve your software by any means. The following are just a few highlighted reasons for having an API built into your application.

  • Separation of functionality. A programming concept any software engineer should know well is the separation of functionality. You want your user interface, persistence, and user/business logic to be in separate layers. PHP has been notorious for making it easy to violate this idea. So many php applications have their user interface, persistence to sql, and other user logic all in the same class. By making an API and having your own application access it for data helps separate the parts of your program.
  • Easily access application on multiple devices. This is probably the biggest reason to have an open API. Say for example you are making some type of SAAS application. About half of all web traffic is through high end mobile devices today. If you build your application and it’s just for a desktop you are losing lots of potential clients and traffic. By having an API you can make your desktop website UI and IOS or Android mobile application call into the same API. This means that you can make as many UI’s as you want and not have to rewrite the persistence layer or business logic for each application.
  • Makes you money. As we mentioned before, most large applications have some type of an api available to people for free, but it’s limited. Some companies can make a lot of money through their APIs by charging for high usage. Having an API for your application means you can earn money from people who will want to access it directly.
  • Allow other people to access and contribute to your data. A good API makes your data collection bigger because people will be contributing information to your database. We live in an age of big data. The more data you have the more you can mine it. Data mining is seriously awesome and can also contribute to making more money. For example you can take customer information and show based off of input information if they are likely to buy your product or not. You can even take it a step further and determine what would be needed to most likely close deals.

Various examples of how to build an API

Ultimately if you decide to add an API to your business application you need to do what is best for you and the company who owns the application. If you wrote all your business logic in PHP there are some great rest frameworks for doing a PHP REST API. If you have your business logic in C# then that is what you should use to program your API endpoints. The following are just a few of the industry leading frameworks for making an open REST API.

  • Javascript Node. I have not always been a fan of javascript, but node has been kicking ass with Express. Express is a very easy framework for making a REST API. PayPal made the conversion from Java to Node and here are the measured response time differences. Another thing I have come to love about using Node with Express is the amount of lines of code I need to use. I made a simple recipe application. One backend was in Spring and the other used Node’s Express. The total number of lines in the Spring application was about 3000 and the total number of lines in Node’s application were about 300. Spring requires a lot more verbosity because all DTO (data transfer objects) have to be defined – even when using a noSql persistence layer.
  • Java Spring. Even if I am slowly being convinced of how awesome NodeJS is a lot of places still use Spring or are in the process of converting to spring. Spring has a REST Client  framework that makes creating a rest application super easy and simple. Spring is all in Java and with that you can write your application in either Scala or Groovy too. I still love the Java language and Spring takes out so much verbosity in building an API.

There are plenty of examples in both node and spring on how to build an API. The body of the requests is most easily done using JSON, but XML or plane text can also work. Just remember that best practices dictate the following REST requests.

  • GET is used to get data from your application. Use this for a single object or a list of objects to return.
    • GET: www.domain.com/api/object/id
  • PUT requests are used to upsert data. If data doesn’t exist it creates it, and if it already exists it will update it.
    • PUT: www.domain.com/api/object
      BODY: { "id":"12345", "foo":"bar"}
      • This example takes an id should find the ID in your database and update the field “foo” and set the value of “bar.” Normally you return the full object with the id. If there is no object the response will return the id of the newly inserted object.
  • POST http request is used to create a new object.
    • POST: www.domain.com/api/object/
      BODY: {"name":"Bilbo Baggins", "age":"111","race":"hobbit"}
      • This type of request should insert a new object into the database for Bilbo Baggins and the response should return either the full object or just the ID of the newly inserted object.
  • DELETE removes the persistence of the object – deletes it from the database or adds a ‘deleted’ attribute to the entry in the database. Big data never really should delete date, only insert and modify it.
    • DELETE: www.domain.com/api/object/id
  • OPTIONS are used to get the available list of options you can use with a method or object.

There are other types of http requests that you can use in your rest application, but these are the most common. I have seen applications where you go to delete the object and you did a post with a JSON body that said delete = true. This is terrible! You should simple just use http DELETE to  api.application/object/id in order to remove an object. Using the HTTP standard requests will actually save you network traffic as the scope of what is carried in them is limited to the function they are performing. This is great when you make a mobile application and all data being transferred is precious.


API Endpoint Testing With Groovy & Spock

best endpoint test frameworkThere are several ways of making http requests through java and groovy. You can use anything from Netflix’s feign client to standard http url connection classes built into java. I have used many different classes/frameworks for testing system integrations. My favorite choice by far is to use either groovy’s http builder or another class that extends it called RESTClient. Typically I build a wrapper class that logs into the service and stores server authentication credentials. This wrapper class works best as a singleton or using dependency injection to keep only once instance of the class available for use throughout the test suite. The reason I love groovy’s restclient is because it is so easy to make the standard http requests for any different type of endpoint. You can then make objects that represent the basic crud operations to wrap your api’s endpoints. Next you make a bunch of Spock specifications making great endpoint testing specs.

Wrapping the RESTClient

The following is just an example of wrapping the rest client into a singleton. When the object gets initialized you can log into your api. Making this a singleton is great because you can just store the authentication credentials and every request you make doesn’t need to reauthenticate. If you ever need to store various user roles you can just store the authentication keys in the singleton too.

package util

import groovyx.net.http.HttpResponseDecorator
import groovyx.net.http.RESTClient
import spock.lang.Shared

 * Created by derekcarr on 5/14/15.
@Singleton(strict = false, lazy = true)
class RESTClientHelper {

    protected RESTClient rc
    @Shared def authResp
    @Shared def userName = "user.name"
    def host = System.getProperty("test.host","www.domain.com")
    //This is to initialize your rest client with the authentication you need
    private RESTClientHelper(){
        authenticationSID = login(host)

    public synchronized HttpResponseDecorator delete(Map vals) {
        return rc.delete(vals)

    public synchronized HttpResponseDecorator get(Map vals) {
        return rc.get(vals)

    public synchronized HttpResponseDecorator put(Map vals) {
        return rc.put(vals)

    public synchronized HttpResponseDecorator post(Map vals) {
        return rc.post(vals)

    void dump(HttpResponseDecorator resp) {
        println 'Header/Status\n--------------------------------'
        println 'Status: '+resp.status
        println 'Content Type: '+resp.contentType
        println 'Headers:'+resp.allHeaders
        println '\nBody\n--------------------------------'
        println JsonOutput.prettyPrint(JsonOutput.toJson(resp.data))

    private addHeaders(Map vals) {
        if (vals.containsKey("headers"))
            rc.headers += vals

    def login = { host -&gt;
        //Login and get a SID
        def u = "qa14.tester@domain.com"
        def p = "p@ssw0rd"
        def resp = new RESTClient("https://${host}").post(
                path: '/api/domain/auth/login',
                body: [username: u, password: p],
                requestContentType: ContentType.JSON
        return URLDecoder.decode(resp.responseData.sid.toString(), "UTF-8")


Internet Explorer Bashing

Internet Explorer eats glue

The first development job I had was working for BYU’s Independent Study program converting lessons into website material. The bane our existence was Internet Explorer. Anybody who has developed javascript a few years ago knows that every version of IE had a different engine. The css is also rendered different for each version of IE. We would have to find ways adjust for each version of IE. Below are several comics I have seen and collected over the years bashing internet explorer – hence the title Internet Explorer Bashing.

If each browser had a cartoon representing its awesomeness.

If a cartoon character had to represent each browser - ie would be a brain dead monkey

Be the designated friend who doesn’t allow his or her friends to use IE

ie warning signs

Almost as bad as being picked last for a sports game.

no party for ie8

This rings so true!

slow response internet explorer bashing

This one is probably my favorite – reminds me of Ralph from The Simpsons

Internet Explorer eats glue

Stupid iPhone keyboard layout

I have a terrible beef to pick with Apple Inc. I originally started switching to Apple’s products years ago because they spent time to quality control their hardware and software. I was so tired of spending time fixing my development environment just so I can write code. The iPhone was a natural progression of jumping on this bandwagon. There are still several things that I love about my iPhone, but there is one default keyboard layout that is absolutely atrocious! Starting off most of us know what muscle memory is. Piano players have it when learning songs or site reading music. In order to site read music well a pianist needs to relate where the note on the sheet music is in relation to where his or her fingers are on the piano’s keyboard. The more a pianist plays the piano the more they know where the fingers go just due to muscle memory. Sending texts or SMS on a phone is no exception. People can type faster as they build muscle memory. Well Apple went and screwed this one up!

horizontal stupid iphone keyboard layout

This is an iPhone 6 keyboard in landscape mode. Observe how the emoticons are on the left and the numbers are on the right. If I always type in one layout this is fine, but contrast that to the other photograph of the keyboard in vertical mode.

Vertical stupid iphone keyboard layout

This is the iPhone 6’s keyboard layout in vertical mode. Notice how the numbers and emoticon positions switched from the horizontal or landscape layout.

I don’t know what usability tests Apple did in deciding to make one layout a certain direction and change it for a different layout. How am I suppose to be fast at texting or taking notes if the stupid keyboard layout changes every time I change my phone’s orientation by 90 degrees. The person who decided this should be fired.

Using groovy closures in Spock where blocks

Groovy Spock vs Java jUnit

Because Spock is just jUnit under the hood we can do everything we can in jUnit and more because Spock uses groovy. One aspect of the jUnit test framework that most people don’t take advantage of is using parameterized classes – making the test class be test driven. The best part of this is you can run the same test with different variables, allowing all sorts of edge cases to be tested without having to rewrite the test class. Spock makes this visually very easy by using a where block. The first line of the where block is the variables that you want to use through out the test. The second line (and every line after that) is a new set of data that you want to test. Remember the test will run as many times as there are lines in the where block.

class testWithClosures extends Specification {

  def setupSpec() {
    //Runs before the class initialization 

  def cleanupSpec() {
    //After the class tests run this will run

  def "Some super awesome test with closures"() {
      Some conditional setup

     specialAssertion | _
     { Whatever Code you want to special validate } | _ 
     { Another special case validation }


The blessings of using Groovy with Spock

Using Groovy in Spock has one amazing perk. The example above lets you define a variable in the where block called specialAssertion. You can take advantage of closures in groovy by placing closures inside your where block and then calling them in your spec class. I occasionally use this if I want to do some special type of assertion with the data that I am passing to the spec. You don’t have to do assertions, but any type of code that needs to be called just for that line of data in the where block. This can be thought of as a similar polymorphic approach of abstract classes and methods.

What is a QA tester?

The many types of QA testers

What is a qa tester? What is a QA tester? First off QA stands for quality assurance and if you are a QA tester you are testing the quality of any product to make sure it meets a certain standard of quality. There are several different types of QA testers in the world. Call centers, for example, will have a QA tester to periodically listen to customer service calls and see if the calls meet the company’s definition of quality. Another example would be in a manufacturing line. A QA tester would take random samples of a product being produced and determine if the quality is good. In a world where information is shared so rapidly people can go online, find the product they have purchased, and rate what they think of it. The point of a QA tester would be to find problems before the user does. You will not find every problem, but if you can mitigate major problems then when people review your product you will not have as much negative pr (public relations). Consumers will write something bad about a product far more then they will write a good review.

Software QA testers

While there are several types of QA testers in the world, the most common today is in the world of software engineering. If you talk about a quality assurance tester for testing software there are two main paths of testers.

  1. Quality Assurance Engineer – The qualifications for a QA engineer are the same for a software engineer. He or she has to know how to write code and test code. In a nutshell a QA engineer programs tests to validate software, works on frameworks to help a software developer test his or her code, and makes sure that the pipeline from creation to production of software has automated validation of the quality. This type of engineer can also be more focused on testing how the software behaves under stress or load conditions, or programmatically find vulnerabilities in the security of the software being developed.
  2. Quality Assurance Analyst – This job position does not necessarily require advanced knowledge of programming, but rather an in depth knowledge of how the product behaves and functions from a user’s perspective. An analyst is primarily focused on exploratory testing and seeing how a user would navigate his or her way through the product.

Make Your Own Website Scraper

Description of a website scraper

I recently did a project where I needed to scrape a subset of webpages from the internet. Back in the 1990s all websites were pretty much straight up html, which makes crawling them and getting their data pretty easy. All you needed was to grab the html files and run them through a tokenizer, parsing out all the juicy bits. Today things are a bit more complicated as most websites have some form of javascript execution (usually jQuery or AngularJS). In order to crawl these type of sites the web scraper needs to be able to execute javascript. There are several ways this can be done each with their benefits and drawbacks. I chose to go with Selenium Webdriver

Of the various implementations of the webdriver – chrome, firefox, internet explorer, htmlunit, and etc . . . the fastest is chrome by far. The basic idea of a web scraper is to load a page as the user would see it. Grab the text from the page that you need, massage it, and store it off somewhere for data mining later. In the process of parsing the pages you grab all the anchor tags and push them into a list of websites “To Be Crawled” and once you scrape the particular website move them to a list of “Crawled Websites.” That’s a web scraper in a nutshell, however, the implementation of one can get a bit more complex especially depending on what your goals are.

One of the major problems I ran into quickly was how angular routes store references to pages. They use a baseurl.com/#/rest_of_link. The reason this is bad is because you need to strip the “#” hash sign from other url’s like foo.com/page#div, which references an id of an html element to scroll to on the loading of that link. The following is the regex I used in order to do strip out the #! and #_end_of_url. It uses the regex look ahead to accomplish the goal.

List<WebElement> list = driver.findElements(By.xpath("//*[@href]"));
for (def e : list) {
    def href =  e.getAttribute("href");
    if (href.contains('#')) {

        href = href.replaceAll("(?!#!)#.*","")

        UrlModel newModel = new UrlModel(href);
        //SUBMIT TO DB

Then once you have a list of url’s to crawl you can just initialize a new chromedriver instance with the url in question and grab all the webelements in the body. Once you have all the web elements you can grab the text values. The rest is up to you how you want to store the data and mine it later. I did not need this to go particularly fast but you can map reduce the scraping by running multiple instances of the chromedriver on several different machines. Selenium grid has a good way of doing this.

Web Designer Job Description

Web Designer Job Description

Years ago if you were a web designer that meant that you knew how to arrange a page in HTML, change colors, and work on a little bit of javascript. With such modern advances with html5, jQuery, and AngularJS, the frontend of web development needs people who are better skilled at user interface logic and can architect massive projects. Although a company may have a specific stack or set of libraries for their front end logic, there are some skills that a good web designer needs to have and companies should look for individuals with these qualities. The web designer job description is slightly different than a typical software engineer because the web designer generally has to live within the constraints of a web browser. This does not make solving problems easier. In fact because each browser behaves differently this job can be seemingly more complex.

GOOD Web Designer Description

  • UNDERSTANDS NUANCES BETWEEN BROWSERS: This is where experience really counts. Each browser has its own javascript and css interpreters and this causes code to have to be written conditionally for each browser. With html5 and most new versions of browsers this has become more unified, but Internet Explorer for example is notorious for not even being backwards compatible with itself version to version.
  • UTILIZES HTML5: HTML5 has eliminated a lot of legacy design patterns. Nobody orders a webpage by tables and rows anymore, instead it is done with divs and appropriate css in order to better accommodate the vast differences in users
  • APPROPRIATELY ABSTRACTS CSS: There are many tools out there that help with css abstraction such as compass-style or sass. While these tools are useful, being able to just know various css properties and how to abstract classes is vital. Abstraction in css will make a web site easier to maintain and update to a newer look in the future.
  • CAN CODE PROFICIENTLY IN JAVASCRIPT: Several years ago javascript was viewed as awful and people were encouraged to turn off javascript in their browsers. Today most webpages use javascript in some form or another. One of the biggest benefits of using javascript is a user will download the page to their browser and all the computations happen on the individuals computer rather than the server. The only requests then go to the server to retrieve data or documents.
  • UNDERSTANDS ASYNCHRONOUS PROGRAMMING: This is probably one of the most important parts of a good web designer. Knowing javascript is good, but the requests are asynchronous. Normal programming paradigms fall apart here. A prime example is making http requests in javascript. A good web developer knows what a promise is and how to use callback functions to process multiple http requests that are dependent on happing in order.
  • KNOWS REST CONTRACTS: Projects that have large data or large number of users often abstract logic into front end and backend code. The front end logic will request data or documents from the server, and the server will query for them and return them in the form of json, xml, or other various serialized forms. Being able to know how to send requests and process response bodies is vital for a web designer. This also means knowing how to handle form submissions and normalizing data from a user that is acceptable to a server.
  • ABSTRACTS FRONT END CODE: Abstraction is a reoccurring theme in most software. User interface code is no exception. Angular has a good example of having a standard MVC (Model View Controller) style of abstraction. Each class or object has a purpose and it makes maintaining code much easier.
  • CAN SEARCH GOOGLE: Knowing what to search for is valuable. Sometimes websites have very weird errors that are not very explicit. Looking at a javascript stack trace may show where a line broke in some library, but not what line in your code broke it. Knowing what to search for to solve problems only comes with experience. The more projects a web designer has worked on the more exposure to the various problems that will occur will be more familiar.

BAD Web Designer Description

  • WEB DESIGNER IS NOT A UX DESIGNER: While having a web designer that can make a page look good, it is not his or her job to necessarily design all the functionality, layout of the page, or pick a color scheme. Web designers that freelance generally will do this, but when you start getting involved in a production software team having a UX designer to help facilitate colors and what layouts will attract users is different than the actual implementation of the user interface.
  • FALLS BEHIND WITH TECHNOLOGY: As this is true of any computer career, having a web designer that does not know how to make a webpage viewable by screens of different sizes, including mobile devices, can drastically impact the amount of visibility a website will receive. Google now calculates a website’s ranking in their search engine by being easily viewable through mobile devices.
  • LACKS SCALABILITY: Any person can write an html page and make several static html pages. When you start working on large projects making plain html pages does not scale well or fast. For example – if you make a recipe site, entering all the data as html is tedious and doesn’t promote involvement from other people.


As with most things in life you pay for what you get. If you want to offshore website development or find some high school free lancers, you can get a project started. If you need something that has a lot more involvement, such as producing graphs or using a html5 canvas, finding someone with more experience and a better sense of software abstraction and development will help make your project more successful, scalable, and generally better for the users that will interface with it.