The State Pattern

The state pattern is useful when an object’s behavior is determined by its state given sufficient complexity. Events trigger the transition of a state to another. A formalized construct that manages the transitions is called a state machine.

To implement the pattern, we need to define:

  • A state entry and exit behaviors
  • Actions when a particular event causes a transition
  • Guard conditions enabling and disabling a transition
  • default actions when no transition is required for a particular event

For example, companies often used automated phone system to redirect and filter calls. We are given choices and depending on the choices, we are given specific answers or redirected to a person. This example is perfect for the state pattern. We can maintain a state object of the phone call and create rules that list the different options depending on the choices the caller made or redirect the call to another phone number.

The Challenges of Micro Services

Micro Services are trendy… and not without reasons, but it is highly theoretical. You have probably noticed it as well when you started to design those Micro Services (MS): How do I share code? How do I share data? How do I deal with foreign keys? Should this DB table be part of another MS? How do I deal with cross-MS transactions? etc. In practice, MS have many challenges and this article is trying to hint how to deal with it. There is no perfect solution. It is all about weighing each solution’s trade-offs.

Bounded context

The concept of bounded context is crucial to understand those challenges. MS are a “share-nothing” architecture. It is a Domain Partitioned Driven design. What it means is that MS are divided by business domain. For example, a MS manages a cart (i.e. how products and quantity by customers). A classic architecture usually follows a Partitioned Technical Design (i.e. divided by technical layers: user interface, business logic, database, etc.).

The main idea behind Micro Services is to isolate a piece of code having one single purpose done very well. The bounded context is belong to the MS’s domain. In our cart example, it includes the quantity of products by customer. It excludes product and customer information.

Challenge #1: Dealing with code reuse

What do we do with shared code? Simple example, each MS needs to authenticate who’s using the service. How do we share the code? Programmers live by the “reuse is abuse” principle, but how does it work with the MS’ “share nothing”? Maybe Micro Services imply “Please Repeat Yourself!” (PRY). Here are a few ideas:

  1. Why not copying the code if it does not change and if there is no bug. The problem is that fixing bug is difficult! and difficult to expand (add additional classes)
  2. Shared library (jar, dll, etc.): can fix and add more classes, but managing versions is hard. Many MS depends on it, which means every MS would need to upgrade the library. And what about heterogenous code base (MS can have be coded in different languages)? In that case, shared library cannot be shared.
  3. Shared service: positive, high code volatility (make changes without impact), heterogenous polyglot code (MS’s language does not matter). However, versioning is difficult (runtime changes: down times can break all the other MS). Performances can be slow due to network latency. Availability may not be guarantee, etc.
  4. Service consolidation. One way to avoid shared code is not to have shared code! Basically, it is merging MS that need the same code, but there is larger testing scope, greater risk to deploy, less agility…

Rule of thumb: Make shared code unit as small as possible -> trade off maintenance (e.g. how many MS need to be updated) for governance (e.g. clarity in what shared code each MS depends on). Smaller units mean more dependencies (e.g. MS1, MS2 have the same shared code A. MS1, MS3 have the same shared code B, etc. vs MS1, MS2, MS3 have one shared code ABC), but updating shared code impacts less MS (e.g. Only MS1 and MS2 need to be updated if shared code A is updated vs MS1, 2, and 3 need to be updated if ABC is updated).

Also, who owns the shared code? Ideally it is owned by the primary service domain.

Challenge #2: Service Granularity

Drivers for MS:

  1. Code volatility (frequency of change)
  2. Fault tolerance: if a service crashes, what happens? Does it make sense to split to keep a service up?
  3. Scalability and throughput: easier to increase resources without changing the code.
  4. Security

Factors for granularity:

  1. Service functionality: service scope and function: single-purpose function.
  2. Database transaction: how to sync data between different MS? ex: bank account. The transaction must commit or rollback, not be unstable.
  3. Data dependencies: how to break the data? Multiple MS may need data from different tables. Cannot do SQL joins! MS need to talk to get data: performance goes down, latency increases, etc.
  4. Workflow and choreography: everybody is calling everybody. What happens if one MS goes down? How does it impact the other MS? Does it end up taking a whole bunch of MS down / not working / timing out? Performance is going down because of latency. MS Security also increase latency. Also there may be data inconsistency. Updating multiple MS implies too much communication. Too fine grain MS. Minimize amount of inner service communication (Microservices Death-star AntiPattern).

Rule of thumb: get granularity right by iterating: from more to less.

Challenge #3: Sharing Distributed Data

Once we design the MS, how do we share the data? ex: wishlist service and product service: wishlist needs to get information from the product services.

  1. Call the service: increase latency (100-300ms with security).
  2. Replicate the data in the wishlist: possible data inconsistency. Worse!
  3. Caching: replicating data behind the scene in the wishlist. It is more fault tolerant if the product MS goes down, and faster. But volume is an issue, and update rate might be too high leading to data consistency. It may work if the size and the number of updates are low.

Another solution: Creating Data Domains! It is possible to bend to MS rules by sharing a schema when a few MS (2-6 MS) are closely related. It is a pragmatic approach to the MS theory. Otherwise, we would need to break most foreign keys, views and stored procedures. With creating data domains, we reduce the number of breaking. We are mapping information, not table.

Challenge #4: Communication Challenges

MS communicate through API gateway, load balancer or reverse proxy. Stamp coupling (i.e. Passing data structure with more data than needed) is the problem. Ex: retrieving 45 attributes for one needed. It can quickly increase the use of bandwidth (e.g. 1Gb vs 400 kb)

  1. Field selector: cheap but not effective. API including a parameter specifying what part to return. Not solving stamp coupling since it requires changing the MS contract.
  2. GraphQL Server is a solution instead of API gateway. It implements consumer driven contract. It decouples contract from what is needed: for example get the client’s address instead of the whole profile. It reduces traffic between the GraphQL server and the client MS, but still takes much bandwidth from the MS responding the request to the GraphQL server (filtering is done in the GraphQL server).
  3. Direct messaging between the MS solves both stamp coupling and bandwidth. It creates an Internal API without passing through the API gateway. Not going through the gateway. However, what about security, auditing, etc. performed the gateway?
  4. Value-driven contracts: every single contract is simply a map (pair of key/value). The contract’s purpose is to inform the client code what the services offer. In value-driven contracts, the contract is a map, but it implies no structure and no field definition. So the idea is for the client MS to have tests that verify if the producer MS still provides what is needed. If so, the MS can be deployed. “Test” is also “pact.” It solves both issues (bandwidth and stamp coupling)
  5. Replicated or distributed caching: issue with update frequency and volume.

Challenge #5: Orchestration vs Choreography

Orchestration requires a conductor. Choreography does not. They communicate with each other instead.

Orchestration pattern

  1. Gateway is the conductor(s). Multicasting logic is in the conductor. Gateway (ESB) manages client requests, security, id generation, metrics, auditing, service discovery, etc. Deploying is difficult. Update the MS -> update conductor = update gateway. Breaking bounded context.
  2. The conductor is a MS orchestrator outside of the gateway. It owns the contract. Hot deploy is easy. Abstract the MS for the other applications/ESB. It is not a single point of failure when combined with clusters. An orchestrator is usually stateless.

Aggregation Pattern

MS aggregator are useful when there is a need to look at many data types. For example, we may need to look at 140 MS to see what a customer need. Instead of requesting 140 MS, we request the aggregator for all the information. This aggregator has a cache of all the data from the other MS updated possibly via a queue. It is a band-aid pattern though. Each request has its own aggregator. An aggregator is stateful.

Adapter pattern

MS adapters are new endpoint to communicate with other systems which may not be able to communicate through REST requests. Each adapter is dedicated to a specific system. For example, the adapter convert the REST request into a call to a COBOL system.

Challenge #6: How to maintain data consistency in distributed transactions?

Sagas can be used to compensate a failed transaction. More information on: https://microservices.io/patterns/data/saga.html

The Memento Pattern

If you have worked on a document such as a contract, specifications or a thesis, you were probably glad that your text editor (e.g. Microsoft Office) saved your work regularly and kept a few versions somewhere. That’s the idea of the memento pattern. It allows an object to keep a token representing a state of that object.

Let’s see how the code would look like if we want to have our text editor saving versions.

public class TextEditor {
   // for the sake of simplicity, 
   // we use a string to represent the content
   private String content; 

   public void write(String content) {
      this.content = content;
   }

   public void append(String newContent) {
      this.content += newContent;
   }
}

Now we want to create a memento that will keep versions of our content:

public class Version {
   private String content;

   // the version is read only. We set the content
   // when we create the version.
   public Version(String content) {
      this.content = content;
   }

   public String getContent() {
     return content;
   }
}

The TextEditor class needs a new method to save and restore a version:

public class TextEditor {
   private String content; 

   public void write(String content) {
      this.content = content;
   }

   public void append(String newContent) {
      this.content += newContent;
   }

   public Version save() {
      return new Version(content);
   }

   public void restore(Version v) {
      this.content = v.getContent();
   }
}

The code that uses the text editor looks like:

public class Application {
 
   public static void main(String[] args) {
      TextEditor editor = new TextEditor();
      editor.write("this is my first version.");
      Version v1 = editor.save();
      editor.append("I am adding a second version");
      Version v2 = editor.save();

      // restore version 1
      editor.restore(v1); 
      // content = "this is my first version."

      // restore version 2
      editor.restore(v2); 
      // content = "this is my first version.I am adding a second version"
 }
}

The main inconvenient to this pattern is the use of memory. If memory is an issue, we may want to use the command pattern which saves commands instead of states (so less memory though more complex handling of the state).

The Mediator Pattern

The Mediator pattern facilitates communication between components without having references to each other. It avoids tight coupling between components and helps maintainability.

Let’s say we are building a user interface to visualizes photos. There are multiple components such as the image, the title, the author, the image exif info, the comments, the thumbnail bar, etc. When a user selects a new image, all those components need to be updated. Without the Mediator pattern, the “next” button would call all the components to update them.

public class NextButton {

private Display imageDisplay;
private Label title;
private Label author;
...

public onClick() {
// some code to get the image
imageDisplay.goto(image);
title.set(image);
author.set(image);
// etc...
}
}

We can also imagine that adding a comment would update the title of the image with the total count of comments. It would mean that the comment section has a reference on the title. All those components would be tightly linked.

The Mediator pattern introduces a new component which would be the only one with references to all the different components. Whenever a component needs to communicate with the others, it would send a message to the mediator which then dispatch it to the others.

In our case, the click on “Next” would send an event.

public class NextButton implements UIComponent {

 private EventBroker mediator;

 public void onClick() {
     // some code to get the image
     mediator.publish(new Event("updateImage"));
 }

 public void listen(Event e) {
   // do something when necessary
 }
}

The mediator would then publish the events to all the components that are listening.

public class EventBroker {
    private List<UIComponent> components;

   public void subscribe(UIComponent component) {
     components.add(component);
   }

   public void publish(Event e) {
     components.foreach(it -> it.listen(e));
   }
}

The Iterator Pattern

The iterator pattern allows the client code to traverse data collections without exposing their structure. The client code does not need to know how the data are stored, and how to go from one element to another.

This pattern usually defines an Iterator class which keeps a reference to the current element and is able to go to the next element. Java has its Iterator interface that many collection classes implement.

Iterator<Car> it = mylist.iterator(); // retrieve iterator object from the collection
while (it.hasNext()) { // browse all the elements
   Car car = it.next(); // return the current element and move on the next element
   // ... do something
}

The Interpreter Pattern

The interpreter pattern evaluates textual inputs. It does it by turning it into separated lexical terms (lexing) and interpreting each sequence (parsing). You can think of programming languages, or languages meant to carry information such as XML.

For example, we want to calculate the result of this formula: 1+3+5*7

First, we need to find the terms: [1,3,5,7] and the operations: [+, +, *]

Second, we interpret them into a logical sequence: add(add(1,3),multiply(5,7)).

The idea is to single out terms and operations in order to simplify and make the design flexible to different inputs (ex: term order, etc.)

How to get rid of SearchMine.Net on Safari

I hate to confess, but I made a terrible mistake… and it is embarrassing to write about it. The other day, I was looking for a math worksheet for my daughter. I went on a legit website, and clicked on the image to print. I was redirected on a page to update Adobe Flash. I never do that kind of things. It always smells fishy. But this time, my daughter is next to me whining. My son is playing with his noisy electric circuit. I have a online meeting coming up soon. I didn’t think too much… and I clicked on “update!” As it started to do things, I was thinking “not good…” I tried to cancel, but too late. Safari redirected me to searchmine.net.

Hate to see this screen…

I was hoping it was not a big deal. I googled how to remove searchmine, and did a few things: get rid of any weird application, cleared caches, private information, extensions, looked at homepage, installed Malwarebytes, Avast, ComboCleaner (free to scan, but not to remove threats. What a scam! Hey ComboCleaner, if you’re not free, just say it!). SearchMine was still there.

Last thing to do: remove Mac OS profile configuration. BUT I was not able to do it. Profiles was greyed out:

Cannot use the UI to update profiles.

Only solution: use command lines since I was admin on my machine. First, list the profiles:

sudo profiles -L

The list was like:

_computerlevel[1] attribute: profileIdentifier: some sort of ID
_computerlevel[2] attribute: profileIdentifier: some sort of ID
_computerlevel[3] attribute: profileIdentifier: com.mycouponsmart.safari.XXXXXXXXXXXXXXXXXXXXXX
_computerlevel[4] attribute: profileIdentifier: some sort of ID

Finally I see the culprit… To get rid of it, I removed it with this command line:

sudo profiles -R -p com.mycouponsmart.safari.XXXXXXXXXXXXXXXXXXXXXX

And that’s it! no more searchmine! Here are the lessons I have learned:

  • Hate Adobe Flash!
  • Hate myself for clicking on “Update.” I should have followed my instinct… If needed, go to the official website to download what you need.
  • Never assume you are too smart to avoid malwares.
  • Never assume Mac OS is free from malwares.

The Command Pattern

The Command Pattern is useful to implement an instruction to perform a particular action. It contains and encapsulates all the information necessary for the action to be taken. It can allow to create composite commands, to keep track of a sequence of commands, and possibly to undo one of the commands. It can be used in GUI or transactions for example.

Think of a text editor. We can type characters and format them. Each command will implement an interface:

public interface Command {
   void perform();
   void undo();
}

public class Write implements Command {

   private TextEditor editor; // we will assume this class exists.
   private String string;
   private boolean success;

   public Write(TextEditor editor, String string) {
      this.editor = editor;
      this.string = string;
      this.success = false; // by default, a boolean is false, but it does not hurt to make it explicit!
   }

   @Override
   public void perform() {
       editor.write(string);
       success = true;
   }

   @Override
   public void undo() {
      if (success) { // we do not want to undo if the operation failed
        editor.erase(string); // more complex in reality, but not the purpose of this example.
      }
   }
}

public class Formatter implements Command {

  public enum Format {
     BOLD,
     ITALICIZE
  }

   private TextEditor editor;
   private Format format;
   private int startPosition; 
   private int endPosition;
   private boolean success;

   public Formatter(TextEditor editor, Format format, int startPosition, int endPosition) { // could chain with Write.. Chain of responsibility pattern!
      this.editor = editor;
      this.format = format;
      this.startPosition = startPosition;
      this.endPosition = endPosition;
      this.success = false;
   }

   @Override
   public void perform() {
      switch (format) {
         case BOLD:
           success = editor.bold(startPosition, endPosition);
           break;
         case ITALICIZE:
           success = editor.italicize(startPosition, endPosition);
           break;
         default: 
           break;
      }
   }

   @Override
   public void undo() {
      if (!success) return; // we do not want to undo if the operation failed
      switch (format) {
        case BOLD:
          editor.unbold(startPosition, endPosition);
          break;
        case ITALICIZE:
          editor.unbold(startPosition, endPosition);
          break;
        default:
          break;
      }
   }
}

The client code can chain commands or undo, etc.

TextEditor editor = new TextEditor();
Command awesome = new Write(editor, "This is awesome!");
Command author = new Write(editor, "By John Doe");
Command formatAwesome = new Formatter(editor, Formatter.Format.BOLD, 8, 15); 
Command formatAuthor = new Formatter(editor, Formatter.Format.ITALICIZE, 16, 27);

List<Command> cmds = new ArrayList<>();
cmds.add(awesome);
cmds.add(formatAwesome);
cmds.add(author);
cmds.add(formatAuthor);

try {
  cmds.forEach(Command::perform); // run
} catch (Exception e) {
  Collections.reverse(cmds); // reverse the command sequence.
  cmds.forEach(Command::undo); // will undo only those that succeeded.
}

This pattern can be combined with the Chain of Responsibility pattern or the Strategy pattern.

This is awesome! But is it really useful you could ask? What about just calling TextEditor’s methods? True! I was wondering the same. What is really powerful is how we can manage those commands. It can be more complex commands, sequenced, and we can create rollbacks, etc.

The Chain of Responsibility Pattern

The Chain of Responsibility pattern is useful when there is a chain of components that need to accomplish the same operation. Each component may have a default implementation and/or cancel the processing chain.

Let’s say we want to write a class that handles http requests. For each request, we want to log it, verify its authentication, and parse the content. Each operation can be designed as a class implementing the same interface.

public interface HttpRequestHandler {

   HttpRequestHandler next();
   void setNext(HttpRequestHandler next);

   // allows to chain other handlers
   default void add(HttpRequestHandler handler) {
      if (next() != null) {
        next().add(handler);
      } else {
        setNext(handler);
      }
   }; 

   void process(HttpRequest request);
}

public class Logger implements HttpRequestHandler {

   private Log log = ...
   private HttpRequestHandler next;

   // implements getter/setter
   // ...

   public void process(HttpRequest request) {
      log.log(request); // order can changed if we want the operations to start at the end of the chain
      if (next != null) next.process(request);
   }
}

public class Authenticator implements HttpRequestHandler {

    private HttpRequestHandler next;

    // implements getter/setter
    // ...

    public void process(HttpRequest request) {
      if (authenticate(request)) {
         if (next != null) next.process(request);
      }
      // the chain is disrupted if the request does not pass the security check
    }

    private boolean authenticate(HttpRequest request) {
      // ...
    }
}


public class Parser implements HttpRequestHandler {

    private HttpRequestHandler next;

    // implements getter/setter
    // ...

    public void process(HttpRequest request) {
      parse(request);
      if (next != null) next.process(request);
    }

    private void parse(HttpRequest request) {
       // ...
    }
}

The good thing with this design is that we can add more operations without too much effort. Now we can have our client code manipulating the handlers:

public class WebServer {

   public HttpResponse respondTo(HttpRequest request) {
      handlers().handle(request);
   }

   private HttpRequestHandler handlers() {
      Logger logger = new Logger();
      Authenticator auth = new Authenticator();
      Parser parser = new Parser();
      logger.add(auth);
      logger.add(parser);
      return logger;
   }

}

The Proxy Pattern

The Proxy pattern creates an interface giving access to a resource to hide any underlying-complexity as well as to add improvements of the underlying code. The proxy interface looks like the interface it “caches.”

Let’s think about a query object we want to use to query a database. It is great to log any request for debugging purpose.

public class SimpleQuery<T> implements Query<T> {

   public T findBy(String id) { 
       // connect to database, query, etc.
       return db.query(sql);
   }

   public void insert(T object) {
       // connect to database, create id, etc.
       db.insert(sql);
   }
}

We are going to proxy this with inheritance. It is possible to implement this pattern with other techniques such as composition (having a reference of the object to proxy), or introspection if we want the proxy to be dynamic (done at runtime).

public class ProxyQuery<T> extends SimpleQuery<T> {

   private Logger logger = ... // whatever framework you want to use
   
   public T findBy(String id) {
     // connect to database, query, etc.
     String sql = "select * from table where id="+id;
     logger.debug(sql);
     return db.query(sql);
   }

   public void insert(T object) {
     // connect to database, create id, etc.
     String sql = "insert into table (id, value) values ("+id+","+object+")";
     logger.debug(sql);
     db.insert(object);
   }

}

And that’s all! Of course proxies can be much more complex. It can be used to call a remote object. It can be used to use a cache, etc.

Now you may wonder: what is the difference between the proxy pattern and the decorator pattern?

  • The proxy pattern provides an identical interface while the decorator provides an enhanced interface.
  • The decorator usually aggregates (composition) what it is decorating. The proxy does not have to work that way.
  • Proxy might interface an object that has not been created yet. For example, you may want to stack up / validate information before creating the actual object for performance sake.

Design a site like this with WordPress.com
Get started