Find Part II here

Transferring the liability of errors inherent in the task of writing source code from user to machine is accomplished through automated testing. Used purposefully, test driven development makes writing source code much more enjoyable and pleasurable.

Of course, testing, is not without errors. Test code that is not written well could very well be testing the wrong thing or nothing at all. In this manner, testing incorrectly can give off the illusion that the source code is both functional and reliable when it isn’t.

To understand whether the test code is achieving its intended target, nothing supercedes following a passing test with manual testing. This approach helps the operator to begin interpreting the relationship between automated and manual testing. Consequently, one not only appreciates how automated testing replaces manual testing, but also how to write them. Knowledge of how to write these tests reflect principles that we have unknowningly picked up and incoporated into our understanding. It is these silent and unheard principles which I would like to elaborate upon since they will save newcomers the expense of much frustration and disappointment associated with a trial-and-error approach to understanding testing.

Yes, an understanding of these principles will turn the process of testing from one of moritification to one of delight (or at the very least, tolerance).

First: manual vs automated testing

If manual testing is having a user physically verify each element of the web application, whether it be link or content, checking for functionality etc, then automated testing is having a “command line” version of that. This is the key: after grasping this one core principle, testing becomes much more intuitive to understand.

Where one would change directories in a terminal for shell to interpret, you too would require valid commands to be inputted in order to both get to the correct webpage and test it.

Second: the commands

Like CLI commands, one must understand where one initially starts and where one is headed towards.

Where exactly are we? Well… it’s kind of complex. On a superficial level, we are located as one would be when one fires up a web browser with a blank homepage url. However it becomes obvious when one inspects his surroundings that he is actually inside the test environment.

The ‘test environment’ is an abstract and desolate place with scatterings of small populations throughout. These populations reflect copies or certain mirages of structures we have worked with inside the development environment. These structures include our database, hypertext markup files, cascading stylesheets and rail helper files etc.

It then becomes the tester’s job to connect these discrete structures or populations (while the metaphor sustains) together to see if they are actually behaving as expected.

Following is a single integration test block that is helpful for illustrating the scope of the test-environment. For context, this test is testing for a conventional sign-up on a website:

# app/test/integration/users_sign_up_test.rb
require 'test_helper'

class UsersSignUpTest < ActionDispatch::IntegrationTest

  test "sign up with invalid information" do
    get sign_up_path
    
    assert_no_difference 'User.count' do
      post users_path, params: { user: { username: "hop",
                                         email: "hop@govt!",
                                         password: "",
                                         password_confirmation: "" }}
    end
    
    assert_template 'users/new'
    assert_select 'form[action="/users"]'
    assert_select 'div#error_explanation'
    assert_select 'div.field_with_errors'
  end
end

Here’s the breakdown:

# we are inside the test environment:

class UsersSignUpTest < ActionDispatch::IntegrationTest
  #
  #
  #
end

# we open the web browser

class UsersSignUpTest < ActionDispatch::IntegrationTest
  test "sign up with invalid information" do
    #
    #
    #
  end
end

# we enter the url for the sign-up page
# we send a HTTP GET request for the sign-up page

class UsersSignUpTest < ActionDispatch::IntegrationTest
  test "sign up with invalid information" do
    get sign_up_path
  end
end

# we enter strings within the input fields and click submit
# we send a HTTP POST request to the users url with these form parameters

class UsersSignUpTest < ActionDispatch::IntegrationTest
  test "sign up with invalid information" do
    get sign_up_path
    
    post users_path, params: { user: { username: "hop",
                                       email: "hop@govt!",
                                       password: "",
                                       password_confirmation: "" }}
  end
end

This very point is a decisive moment of the coding process. Do you know what this test achieves?

‘Absolutely nothing’ is close, but would be incorrect. The test block is certainly not verifying anything (keyword: verify) but it is definitely navigating somewhere. The direction of our navigation is determined by the ‘request helper methods’.

The key word (that doubles up as a method) for making a test block verify elements of our web application is assert. When we make an assertion, we make verifications in order to assert that something is true - otherwise it is false.

Back to the test block:

# our first assertion!
# we assert that our User model, which is mapped to our users database does not change in its number of records after submission of a form with those parameters

class UsersSignUpTest < ActionDispatch::IntegrationTest
  test "sign up with invalid information" do
    get sign_up_path
  
    assert_no_difference 'User.count' do
      post users_path, params: { user: { username: "hop",
                                         email: "hop@govt!",
                                         password: "",
                                         password_confirmation: "" }}
    end
  end
end

Another checkpoint to discuss: this is the first mention of another structure in our test environment - ie, a database. This is important to keep in mind because the test database is separate from the development and production environments. The test database therefore contains fake, pre-generated data which you must specify for test to work out of!

There is in fact, another structure that has been ommitted for awhile which is require 'test_helper'. This helper file is a very unique structure because it entails the beginnings of pulling in more remote structures like gems/libraries to help improve our testing or to populate virtual entities. One example of this ‘virtual’ quality (albeit that everyhing in software is already virtual) that you can find in Hartl’s tutorial is how you will have to generate methods within test_helper to mirror a login because the sessions hash does not work inside a test environment directly…

Now for the final rundown:

# pulling in remote objects..
# populating environemnt...

require 'test_helper'

class UsersSignUpTest < ActionDispatch::IntegrationTest
  test "sign up with invalid information" do
    #
    #
    #
  end
end

# Another assertion - this assertion points elsewhere.
# This assertion is looking towards Views instead of the Models as seen in assert_no_difference

require 'test_helper'

class UsersSignUpTest < ActionDispatch::IntegrationTest
  test "sign up with invalid information" do
    get sign_up_path
    
    assert_no_difference 'User.count' do
      post users_path, params: { user: { username: "hop",
                                         email: "hop@govt!",
                                         password: "",
                                         password_confirmation: "" }}
    end
    
    assert_template 'users/new'
  end
end

# assert_select looks at html elements directly
# this assertion tests for the front-end interface of rendered webpages - this means it can potentially test for back-end data being plugged in!

require 'test_helper'

class UsersSignUpTest < ActionDispatch::IntegrationTest
  test "sign up with invalid information" do
    get sign_up_path
    
    assert_no_difference 'User.count' do
      post users_path, params: { user: { username: "hop",
                                         email: "hop@govt!",
                                         password: "",
                                         password_confirmation: "" }}
    end
    
    assert_template 'users/new'
    assert_select 'form[action="/users"]'
    assert_select 'div#error_explanation'
    assert_select 'div.field_with_errors'
  end
end

Summary

Using automated testing in place of manual testing is not without errors. The first call of action is then to understand how to visualise automated testing which very closely resembles a command line interface. Instead of verifying by seeing with our eyes or through multiple clickings, we rely on assertions within the test. However, exactly like visiting a web page to verify content, we must first navigate and make travels in our test-environment via request methods before we can make assertions.

Regular expressions will always continue to trip me up but I think I am finally get a grip on it.

I think it’s really important to follow an in-depth guide for learning regex if you ever want to take it seriously. And the reason for this is that regex is to strings what BEDMAS is to math equations.

What I mean is that there are certain rules in precedence and convention that take place in choosing the most efficient (and least convoluted) regex for a string that are intuitively integrated into BEDMAS.

BEDMAS denotes the order of priority in which math events take place: brackets first, then exponents, division, multiplication, addition and subtraction last. In BEDMAS there are no surprises or subtleties left uncovered - everything has an order. In learning and applying regex though I tended towards stumbling and fumbling about with no clear direction before ‘landing’ on an answer.

Even after acquiring an answer I would be no closer to understanding how regex worked. This is because, with each iteration of a regex, I would be analysing what the regex pattern would be achieving and then trying to work out the ruleset by finding a trend through reverse-engineering. This sort of logic however is erroneous because regex is very specific. The rules for matching in regex doesn’t bend but when you apply it to a string, it gives off the illusion that the regex is working exactly as you perceived it to. Apply the same regex to a different string and the result is not only are there no matches found in the string you are checking but there are also no matches found in your poor brain. Trying to corroborate how regex works by trial and error without first learning first principles is wasted time and effort.

The most approachable way to appreciate regular expressions in Ruby is to use the #scan method which returns all regex matches into an array for use. I initially tried using the #split method on various exercises which is more focused on finding what the string isn’t but they generally end up as failed attempts and noodle code so I will stick to scan now. After inspecting what the array looks like can we deconstruct regular expressions by first principle.

My list of first principles to know for regex are:

• Single character evaluation
• Brackets vs. Character Sets
• Matching
• Single Object Evaluation
• Modifiers

The string I like to use for analysis is an excerpt from Bob Dylan’s poem ‘Do Not Go Gentle Into That Good Night’:

phrase = "Do not go gentle into that good night\nOld age should burn and rave at close of day"

Single Character Evaluation

Whenever you input a regex, if it is not delimited by repetitions or a range you are always evaluating by a single character:

# Delimiter for a single alphanumeric character
phrase.scan(/\w/)
# => ["D", "o", "n", "o", "t", "g", "o", "g", "e", "n", "t", "l", "e", "i", "n", "t", "o", "t", "h", "a", "t", "g", "o", "o", "d", "n", "i", "g", "h", "t", "O", "l", "d", "a", "g", "e", "s", "h", "o", "u", "l", "d", "b", "u", "r", "n", "a", "n", "d", "r", "a", "v", "e", "a", "t", "c", "l", "o", "s", "e", "o", "f", "d", "a", "y"]


# Delimiter for an entire alphanumeric word
phrase.scan(/\w+/)
# => ["Do", "not", "go", "gentle", "into", "that", "good", "night", "Old", "age", "should", "burn", "and", "rave", "at", "close", "of", "day"]


# Delimiter for span of alphanumeric characters ranging from 1 to 3 in length
phrase.scan(/\w{1,3}/)
# => ["Do", "not", "go", "gen", "tle", "int", "o", "tha", "t", "goo", "d", "nig", "ht", "Old", "age", "sho", "uld", "bur", "n", "and", "rav", "e", "at", "clo", "se", "of", "day"]

Brackets and Character Sets with the OR logic operator

(cat|puma) is not the same as [cat|puma].

In fact, the | logic operator is completely redundant in [cat|puma], which is really [catpuma] or any jumbled up sequence of the anagram like [amuptac]. Actually, there’s a duplicate a inside which can be removed: [muptac]

What [] is, is representation of a single character occupying a space. (This concept of space is important as you will see below with ().) Any characters inside the square brackets conveys that this character can be any one of those inside characters. If the regex was /[ne]/ for instance we would get a match like:

"Once upon a time".scan(/[ne]/)
# => ["n", "e", "n", "e"]

# Internally, selection is like:
"O[]c[] upo[] a tim[]"

Whereas (cat|puma) is really what you expect it to be. It searches for the string literal ‘cat’ or ‘puma’. These brackets '()' end up creating a new environment or space for a combination of characters to be evaluated. It’s like a regex within a regex. Regexception:

"The puma is a big cat".scan(/(cat|puma)/)
# => [["puma"], ["cat"]]

# Internally, selection is like:
"The [___] is a big [__]"

What I mean by “space” is that the regex is looking for a particular combination relative to surrouding elements - for example:

"The puma is a big cat".scan(/The\s(cat|puma)/)
# => [["puma"]]

# Internally, selection is like:
"[The][\s][puma] is a big cat"

Another way to compare () with [] is to show that the following regex are nearly equivalent:

"Hello World".scan(/(e|o|d)/)
# => [["e"], ["o"], ["o"], ["d"]]

"Hello World".scan(/[eod]/)
# => ["e", "o", "o", "d"]

Matching

Although the regex in matchers (() brackets) and character sets ([] square brackets) are evaluating the same thing, they are being outputted differently. Each match found with a matcher is placed within its own array within an array which may be important for doing other iterations. While in the character set, each match found is just passed into the array as its self.

In order to fix this we add ?: to the start of the () brackets:

# The output of this...
"Hello World".scan(/(?:e|o|d)/)
# => ["e", "o", "o", "d"]

# ...is now the same as this:
"Hello World".scan(/[eod]/)
# => ["e", "o", "o", "d"]

Hmph. There’s not enough brain juice left in the tank to cover Single Object Evaluation and Modifiers. Another time perhaps.

Find Part I here

I found that just starting this Gossip House Rails app was super difficult. There are though, a few principles that gave me a boost:

• Roll out one feature at a time. An app is about functionality. This is achieved through vertical — not horizontal — integration. It’s a depth-first approach.
• Make the blueprints for your app. Outline what you are trying to achieve. I draw inspiration for this development process through Matthew Inman’s post on how he built mingle2.
• “The ones that win are the ones that ship.” - is a quote I remember reading from Dive Into Html 5. It is a very cool resource about the history of HTML5 and outlines why she is the way she is. Its arcane preface may cause the unawares to overlook its value: don’t! It’s such a valuable resource to learn from.

So after having a think about the resources, I determined that Gossip House was simply an app with Users, Posts and authentication. I then drew a mind-map for all the features I required.

Here’s a short listing of the plan:

• Users –> have many posts
  • sign up
  • log in
  • authenticate (extras!!)
    • activated?
    • password reset
• Posts –> belongs to user
  • new post
    • post to database
    • get id of current user logged in and save as foreign key SQL
  • show all posts
    • each posts have username and message
    • however usernames are not visible if not logged in; this involves something like if current_user == nil, name = "???"; else posts.user_id.name

Well, a web app isn’t very fun without users so off I went to make the first feature: sign up.

From a data point of view, the premise of the sign up feature is about getting new users into the database via the front end. This is the key principle underpinning my workflow. It keeps my compass needle poininting in the right direction whenever I am lost with all the method calls and MVC architecture.

First of is a scaffold:

rails g scaffold username:string email:string password_digest:string

Initially though my scaffold was generated using:

rails g scaffold username:string email:string password:string password_confirmation:string

However for important security measures the former supercedes the latter. I will elaborate later but it’s becoming clear that back end development always come back to data!

Data is power, guard it well.

I then inspect my config/routes.rb file and set my root url through the syntax root 'controller#action' as root 'users#index'. This follows REST url convention. I set the root different to users#new in anticipation for a redirect and flash message.

I actually ended up deleting all my scaffold generator methods to start fresh. Although I did undo the deletes a couple of times to refresh my memory on how the controller actions were supposed to work again (doh).

I initially set up the index and new actions for now:

class UsersController < ApplicationController
  
  def index
    @users = User.all
  end
  
  def new
    @user = User.new
  end
  
end

I am though, in the back of my mind, anticipating the create action which necessarily follows a form submission from the new view. Trying to set the create action now though isn’t following the “flow” of the app. So I avoid it for now.

If I am going to be making a new user, I have to ensure that there are at least some validations of the data being passed:

The syntax for validations follows:

validates :attribute, validation: arguments

I want to make validations for :username, :email. :password and :password_confirmation. Instead of writing out the validations for the latter two, I instead use the helper method has_secure_password which is supplied through Active Model and bcrypt.

has_secure_password is a helper method that supercedes the password and password_confirmation validations in the User model:

• It ensures that both password and password_confirmation attributes are actually present. Bcrypt does this by creating virtual attributes to work with.
• It also ensures that both of these attributes match.
• It then saves this password as a digest within the database which makes it far more secure than a naked password.

NB: the password is NOT encrypted - it’s digested. The difference is that encryption is a reversible action. A password that is encrypted can be decrypted. A digest however, only goes one way. Once you digest a password it can only become poop - that is, a string of unique gibberish. Long story short, trying to find out the original password of a digested password is far harder to do than an encrypted password.

To include the functionality of digests in the rails app, I include gem 'bcrypt' in my Gemfile and then run bundle install again.

My model file ended up looking something like:

class User < ApplicationRecord
  
  VALID_EMAIL_REGEX = /\A([\w+\-].?)+@[a-z\d\-]+(\.[a-z]+)*\.[a-z]+\z/i
  
  validates :username, presence: true,
                       length:   { in: 6..50 }
  validates :email,    presence: true,
                       uniqueness: true,
                       format: { with: VALID_EMAIL_REGEX, message: "is not valid" }
  has_secure_password
  validates :password, length: { in: 6..250 }
end

Now that I had the validations in place. It was time to get a ‘users#new’ form sorted. A lot of it was already generated in the scaffold command via a form_for helper in the views. The way the scaffold sets it up is to render a partial in the new.html.erb view which can be reused in the edit.html.erb view.

However, the form was damn ugly. UGLY.

So I needed to insert a few classes in order for the form to appreciate the Bulma CSS framework I used. Trying to insert arguments into the form helper methods threw heaps of errors. This was because each form helper — assigned to a different type of element and understandably holding different attributes — accepted the arguments differently. So what really helped was having a look at the Rails API docs to see a list of all the Ruby on Rails helpers used in views. There was a convenient answer on Stack Overflow which demonstrated that all these methods were called via ActionView::Helpers. From there it was easy looking up how to include classes in the helper methods.

The styling helped a lot however the error messages were being placed in a very jarring manner. What I wanted was for only the relevant error messages to be beneath the corresponding input fields. What transpired was code that isn’t DRY but works. That’s fine, I’ll refactor it later.

BEFORE - where the initial code which smushed all the errors together:

<% if user.errors.any? %>
  <div id="error_explanation">
    <h2 class="notification is-danger">
      <%= pluralize(user.errors.count, "error") %> prevented you from signing up
    </h2>

    <ul class="help is-danger">
    <% user.errors.full_messages.each do |message| %>
      <li><%= message %></li>
    <% end %>
    </ul>
  </div>
<% end %>

AFTER - where error messages only appear under their respective input field:

<% if user.errors.any? %>
  <div id="error_explanation">
    <h2 class="notification is-danger">
      <%= pluralize(user.errors.count, "error") %> prevented you from signing up
    </h2>
  </div>
<% end %>

<div class="field control">
  <%= f.label :username, class: "label" %>
  <%= f.text_field :username, class: "input" %>
  
  <% user.errors.messages[:username].each do |message| %>
    <li class=" help is-danger">
      <%= "Username #{message}" %>
    </li>
  <% end %>
</div>

<div class="field control">
  <%= f.label :email, class: "label" %>
  <%= f.text_field :email, class: "input" %>
  
  <% user.errors.messages[:email].each do |message| %>
    <li class=" help is-danger">
      <%= "Email #{message}" %>
    </li>
  <% end %>  
</div>
.
.
.

Now that the form is set up, the request it sends is a post to users_path which is routed towards the users#create action. Since parameters are being passed I need to ensure they are whitelisted. This is a security measure to prevent mass assignment of an object which can compromise the database:

class UsersController < ApplicationController
  .
  .
  .
  def create
    @user = User.new(user_params)
    if @user.save
      redirect_to root_url
      flash[:success] = "You have successfully signed up #{@user.username}!"
    else
      render 'new'
    end
  end
  
  private
  
    def user_params
      params.require(:user).permit(:username, 
                                   :email, 
                                   :password, 
                                   :password_confirmation)
    end
  
end

Without strong parameters, creating a new user would be both insecure and tedious:

@user = User.new(username: params[:username], email: params[:email], password: params[:password], password_confirmation: params[:password_confirmation])

With strong parameters, creating a new user would be secure but still tedious:

@user = User.new(username: params[:user][:username], email: params[:user][:email], password: params[:user][:password], password_confirmation: params[:user][:password_confirmation])

With the private method user_params and the .require and .permit helper methods, creating a new user with strong parameters is finally both secure and efficient:

@user = User.new(user_params)

def user_params
  params.require(:user).permit(:username, 
                               :email, 
                               :password, 
                               :password_confirmation)
end

After tidying up the flash message to show a successful sign up, I considered the sign up feature complete. Although… I still need to do something testing… :scream:

‘Gossip House’ is going to be an app with a listing of posts, in chronological order, from all users shown on a page. Names of the users who have made the posts remain anonymous if you are an outsider - that is, if you are not logged in yet.

It has taken a long time to get to this point:

• I have completed many exercises to do with MVC architecture.
• I have learned the basics of SQL commands - long queries terrify me still.
• I have completed 12 Chapters of Hartl and did a few chapters twice over.

Now I am tasked with rolling out this app. It will be super fun but the scale is huge. There needs to be a bit of planning for an overview of the app as it enables me to break the app down into sizeable chunks to be coded. On top of this, I need to be visualise the ‘flow’ or ‘trend’ moving from one point to another. Resources like ‘Users’ and ‘Posts’ will not come into existence until I have connected it between Routes, Controllers, Models, Views. Then I have to also consider concepts like Authentication and Authorization which will involve sessions, cookies, tokens and digests. Oh and I also need to test everything too. Dang.

So this blog series is not really going to be anything elaborate but it will convey my journey in accomplishing this task. Unfortunately it is going to represent the journey in a linear manner (read from top-bottom) instead of showing a mind-map which is probably closer to what is actually happening in my head and my planning.

So here goes:

rails _5.0.0.1_ new gossip-house

Now previously with Hartl’s tutorials I bundle installed a bootstrap gem to use that css framework. But I really do not like how bootstrap nests its boxes so deeply that specificity is thrown entirely out the window. This type of element and class nesting makes customisation a nightmare. I don’t fancy nightmares so out bootstrap goes.

Well, I do need a css framework though, and I am really enjoying Bulma despite it being in alpha or beta – or some other greek letter – version. Basically it is a very young framework, but it is mighty powerful!

So the first thing to note about Bulma is that it uses Sass files. You need to tell Rails that the Sass content needs to be preprocessed before being acquired. But there is a catch with the way Rails’ asset pipeline behaves which I elaborate upon later.

Here are the steps I used to integrate the Bulma framework into the Rails app:

• cd into app/assets/stylesheets
• run npm install bulma

(I ran tree node_modules/ -L 2 --dirsfirst inside /app/assets/stylesheets to show you this lovely tree of the directory structure)

node_modules/
└── bulma
    ├── css
    ├── sass << move this
    ├── bulma.sass << move this
    ├── CHANGELOG.md
    ├── LICENSE
    ├── package.json
    └── README.md

Move the sass folder and the bulma.sass file into stylesheets level - the level above the node_modules folder. Then delete the nodule_modules folder entirely.

Trying to run the rails app at this point should throw an error where Rails doesn’t know what to do with a variable like $background: #eee when rendering the CSS.

This is because the manifest file looks like this now:

/*
 * This is a manifest file that'll be compiled into application.css, which will include all the files
 * listed below.
 *
 .
 .
 .
 *
 *= require_self
 *= require_tree .
 */

It would seem that Rails doesn’t process any language that requires preprocessing unless it:

1. Is @imported into a Sass file
2. Required in the manifest file

This means that we need to change the manifest file so that only a “master” sass file that imports all other sass files is mentioned:

/*
 * This is a manifest file that'll be compiled into application.css, which will include all the files
 * listed below.
 *
 .
 .
 .
 *
 *= require_self
 *= require bulma
 */

However this means that we would have to keep all the css within the bulma sass file (which would be difficult for maintenance later on) or keep adding *= require sass_file_name to the manifest file for each sass file dedicated to a resource such as users.sass.

The best way to go about this is to import the current tree using @import '*'. You can read about it here:

/*! bulma.io v0.2.3 | MIT License | github.com/jgthms/bulma */
@charset "utf-8"

@import "sass/utilities/_all"
@import "sass/base/_all"
@import "sass/elements/_all"
@import "sass/components/_all"
@import "sass/grid/_all"
@import "sass/layout/_all"
@import "*"

Pretty sweet. This is how you would go about implementing a framework into your Rails app without using gems.

Next up… I will have a think about the Users and Posts resources.