Generating Data - Introducing Tasks

The next part of this application I will be tackling is introducing the concept of a Task. These tasks can encompass many different things, either manually instigated by a user or an automated task that the system has created in order to rectify a data integrity issue.

I'll be using the Command pattern for this as logically this is what our tasks will be, a command to change the state of the application in some way. By storing this as its own object I'll have the option later of making these asynchronous as well (for example by using ActiveJob). They can also require user approval prior to executing if the task needs such a workflow. It also separates the logic out of the Model which is the other location this could could end up therefore maintaining the Single Responsibility Principle.

So the base Task class will look like this:

class Task < ApplicationRecord
  audited

  enum status: {
    pending: 'pending',
    processing: 'processing',
    complete: 'complete',
    error: 'error'
  }

  def initialize(attributes = nil)
    super
    self.status = 'pending'
  end

  def call
    processing!
    execute
    complete!
  rescue NotImplementedError, StandardError => e
    self.error = e.full_message
    self.backtrace = e.backtrace
    error!
    raise(e)
  end

  protected

  # Action logic should be implemented here
  def execute
    raise(NotImplementedError)
  end
end

There is now an enumeration for keeping track of the status of a task. The initial status is pending indicating that the task is waiting but has not been initiated yet. A status of processing shows that this task is now executing its workload. And finally there are two end states, it can either be completed successfully, or fail with an error.

To get better support for the enumeration I'll change the data type in the Postgres database as well:

def up
    remove_column :tasks, :status
    execute <<-SQL
      CREATE TYPE task_status AS ENUM ('pending', 'processing', 'complete', 'error');
    SQL
    add_column :tasks, :status, :task_status
  end

A client of our task only needs to call the call method and the task takes care of everything else.

Building a specific type of task requires that I implement some sort of logic in the protected execute method. This should not be called directly by a client.

So here is an example of a task that I need to build. This one creates an initial set of ItemSellPacks using unique entries from a fixed set of names in /lib/assets/item_sell_packs/en.yml

app\models\tasks\initialise_item_sell_packs.rb

module Tasks
  class InitialiseItemSellPacks < Task
    def initialize(attributes = nil)
      super
      self.description = 'Initialise the ItemSellPacks using the standard list that is shown in P+ ' \
                         'when editing a master product (called Item Sell Pack Name there).'
    end

    protected

    def execute
      names.each do |name|
        ItemSellPack.find_or_create_by!(
          name: name,
          canonical: true
        )
      end
    end

    private

    def names
      YAML.load_file(Rails.root.join('lib/assets/item_sell_packs/en.yml')).values
    end
  end
end

Upon initialize I set a human friendly description of this task so it can be shown in the user interface at some point.

And then the execute method needs an implementation. It reads in a file and creates an ItemSellPack if it does not already exist. I think it will make my life easier if all these tasks are idempotent, that is, they can be executed any number of times without affecting the results of the first run. It also means I do not have to consider wrapping the execute code in a transaction so an error triggers a rollback.

Now I'll run this task to check it works:

> t = Tasks::InitialiseItemSellPacks.new 
 => 
#<Tasks::InitialiseItemSellPacks:0x00007fc2727005d0     

 > t.call
  TRANSACTION (0.2ms)  BEGIN
...
...
  Tasks::InitialiseItemSellPacks Update (0.2ms)  UPDATE "tasks" SET "status" = $1, "updated_at" = $2 WHERE "tasks"."id" = $3  [["status", "complete"], ["updated_at", "2022-06-13 12:31:51.257885"], ["id", 1]]
  TRANSACTION (4.7ms)  COMMIT
 => true 

> ItemSellPack.count 
  ItemSellPack Count (1.1ms)  SELECT COUNT(*) FROM "item_sell_packs"
 => 73

That looks like it has successfully executed and created my initial set of ItemSellPacks.

However, I should also write a unit test for this as well:

require 'test_helper'

class Tasks::InitialiseItemSellPacksTest < ActiveSupport::TestCase
  setup do
    @task = Tasks::InitialiseItemSellPacks.create!
  end

  test 'creates item sell packs' do
    assert_equal('pending', @task.status, 'Task initial status is not pending')
    @task.call
    assert_equal(73, ItemSellPack.count, 'Number of ItemSellPacks created is not 73')
    assert_equal('complete', @task.status, 'Task final status is not complete')
  end
end

I'll run the test and see what the result is:

% rails test test/models/tasks/initialise_item_sell_packs_test.rb:10
Run options: --seed 48747

# Running:

F

Failure:
Tasks::InitialiseItemSellPacksTest#test_creates_item_sell_packs [/Users/andrewfoster/rails/catalogue_cleanser/test/models/tasks/initialise_item_sell_packs_test.rb:13]:
Number of ItemSellPacks created is not 73.
Expected: 73
  Actual: 75

It has failed? So what has caused this? It turns out that I still have the scaffolded fixtures in my source code, so that has added an extra two records to the count. So I have a couple of options here; I can adjust the expected to result to 75 to take the fixtures into account OR I can delete the fixtures altogether.

Having the fixtures may prove useful for other tests that are yet to be written so I will lean into fixture usage and make my fixture files more accurate and change the expected result of this test to take that into account.

I also need to write a test for the generic Task model now too as I've added new code to that class:

class TaskTest < ActiveSupport::TestCase
  setup do
    @task = Task.create!
  end

  test 'should raise an error and mark the task as failed' do
    assert_equal('pending', @task.status, 'Task initial status is not pending')
    assert_raises(NotImplementedError) do
      @task.call
    end
    assert_match(/NotImplementedError/, @task.error, 'Task error message should match NotImplementedError')
    assert_equal('error', @task.status, 'Task final status is not error')
  end
end

I'm testing that a task starts in the pending state, will throw a NotImplementedError if the execute method has not been implemented and set the final status of the task to error.

Useful links:

refactoring.guru/design-patterns/command/ru..

en.wikipedia.org/wiki/Idempotence

naturaily.com/blog/ruby-on-rails-enum