Thoughts about SOA and beyond

Preface

Every serious framework has its Collection classes. Why? Because there is no application that does not need List or Hashtable, thus framework designers thought about that and provided a ready implementation, so we can focus on our application needs instead of writing our Lists and Hashtables over and over again.

Yep, thanks a lot. Now how many times have you defined your own fresh implementation for Customer or Order?

A word about SOA

SOA is about contracts. Of course, there is a famous ABC - Address-Binding-Contract, but "AB" are mostly "solved" by tools, meaning we don't waste time to provide them. Most of our time is spent on contracts, i.e. creating the required classes, getting the relevant data and filling it in, sometimes with other services and other times with our business logic and finally calling the required operation. Actually calling the operation is the 'finally' thing, the time was spent on creating the required classes, getting the relevant data and filling it in.

Why does it happen, why the tools do not help us? Because they do not know how! They 'see' those classes as property bags, they know what are the low level basic types of those properties, but they know nothing about their applicative meaning. So we spend best years of our lives to mapping between different types, yet very common in principle, but invented by us or different vendors.

Can you explain more?

Yes, of course. Frameworks did a great job in helping us to describe how my object is passed between the services. Well, thanks, but I also need some help to use that object. I mean that if I have a Person object and a Phone service, I want to be able to give this Phone service my Person and the tools will get the phone # from the person and pass it to the service. Or even more, I want to feed my grid with Persons and Phone service. The grid should understand that one of the Person fields is actually her phone #, render it as a link that when clicked calls the Phone service! And if this field is email and the service is Mail service, the click will open a 'new mail' window or start a chat or whatever, depending on the field and service in use.

As an application developer I need only to tell where are the 'Persons' and where are the services, all the rest should be done by tools, visual tools.

 

Wow, what do you need to get it live?

Actually much, but not too much. I need a common application object schema. This schema should define what is phone #, what is email and what is person that contains email, phone # and other information. This schema should not cover everything, it should provide only the common denominator around which the majority of our lives revolves. Just consider how much services we would be able to build/consume if we just had Person with its contacts (also Persons) with Address, email and may be some ID. Half of Google applications are built just out of this! And of course this schema must be publicly available, reasonable extensible etc, so every vendor will be able to extend it for his own needs; hopefully opening it for public too!

Interestingly that an effort to create this kind of schema was started, just did not get enough attention. It was a part of LDAP specification, see rfc4519 or rfc4524. Most of the examples I talked about can be created with elements defined there! Of course we don't have to evolve LDAP specification, but this can be a good start: standard concept, description language and even good, ready elements and many existing tools.

 

OK, you got your schema. Describe your perfect world!

Suppose we have a reasonable big number of common types.

• There will be a substantially big number of services 'working' with those types. It will be clear to the vendors what API the world expects. The leading platforms, like .Net, will probably provide some basic services, like persistence, out of the box.
• It will be much easier to consume those services, since they will fit each other.
• Workflow engines and their complementary graphical designers will be able to manipulate on a much higher level of abstraction: there can be a concept of Person, Customer, Order etc as basic types with corresponding graphical representation. Probably many services will be also able to identify themselves and get correct glyphs.
• GUI controls will also upgrade and operate with higher level objects. They will be also able to take service references as parameters to get their work done!
• And finally every platform will provide a class library with those common types implemented and having many useful methods. Those types will be really first class citizens in the SOA world and application developers will need only extend them, if required. Their learning capabilities and efficiency will also increase since there will be much more common in this world!

 

This is how I would like it to be, and very interested to hear you opinion.

Please leave your comments!

Method Hiding - Feature or Bug in Compiler?

Since its very beginning C# has a 'method hiding' feature, which lets us 'hide' base class non-virtual methods. In other words one can write:

class Base
{
    public void Test() {
        Console.WriteLine("Base");
    }
}

class Derived : Base
{
    public new void Test() {
        Console.WriteLine("Derived");
    }
}

//...

Derived d = new Derived();
d.Test(); // "Derived" is printed

So far so good, but what happens if I write:

Base d = new Derived();
d.Test(); // "Base" is printed

Is this an intended behavior? Well, may be yes, but may be no. In case of 'no', we just called a wrong function, and no warning or error was issued by the compiler... Just consider again what happened, we declared a non-virtual method - that's our contract that should be ensured by the compiler. Instead it provides us with a feature to break it.

That's not all, consider the situation where you have some 3rd party library and use it in your project. Now the vendor releases a new version where he hides some method and provides an alternative implementation. Your program will still call the old method until you recompile! So in case you have several modules some recompiled and some not, you may have inconsistent behavior...

Of course you can always find use-cases, where the feature is carefully used and actually is very useful in that context. Yet its disadvantages outweight the advantages, so I think it would be better if we will develop our programs without using it.

Unfortunately there is an additional very similar "feature" - private interfaces, which can be implemented by a type even if its base class already implemented it and declared it sealed. Again the language helps us to break the contract and if it's not done carefully enough, the bugs won't wait to come...

Domain Specific Language (DSL) with Workflow, Part 2

In the previous article I presented some basic ideas for creating DSL activities in Workflow. In this part I want to deep into the challenges I faced during the implementation and share the solutions. As usual, let's start from the requirements:

1. DSL is a set of services, each one implemented by a separate function.
2. For each such function provide a specialized Activity.
3. All the return/out/ref parameters should work as expected.
4. Developing of those Activities should be simple and provide Activity validation.

The first requirement is easy - I can have a separate method for each DSL service in my ExternalDataExchange interface:

// The ExternalDataExchange attribute is a required attribute 
// indicating that the local service participates in data exchange with a workflow
[ExternalDataExchange]
public interface ITaskService
{
    int CreateTask(string taskId, string assignee, ref string text);

    //...
}

To meet the next requirements I wanted to provide a base class, that by simple derivation from it and decorating my properties with some attribute, the desired DSL activity will be created:

[ToolboxItemAttribute(typeof(ActivityToolboxItem))]
public class CreateTask : DslActivity<CreateTask>
{
// Properties on the task
public static readonly DependencyProperty ReturnValueProperty = DependencyProperty.Register("ReturnValue", typeof(System.Int32), typeof(CreateTask));
public static readonly DependencyProperty AssigneeProperty = DependencyProperty.Register("Assignee", typeof(System.String), typeof(CreateTask));
public static readonly DependencyProperty TaskIdProperty = DependencyProperty.Register("TaskId", typeof(System.String), typeof(CreateTask));
public static readonly DependencyProperty TextProperty = DependencyProperty.Register("Text", typeof(System.String), typeof(CreateTask));

public CreateTask() 
// Map the activity to the service method
: base(typeof(ITaskService), "CreateTask")
{
}

// Property, mapped to the return value of the method
[DesignerSerializationVisibilityAttribute(DesignerSerializationVisibility.Visible)]
[BrowsableAttribute(true)]
// "Well Known" name for the return value
[DslParameter(Name = "(ReturnValue)")]
[Category("ReturnValue")]
public int ReturnValue
{
    get
    {
        return ((int)(base.GetValue(ReturnValueProperty)));
    }
    set
    {
        base.SetValue(ReturnValueProperty, value);
    }
}

// Property, mapped to the 'assignee' parameter of the method
[DesignerSerializationVisibilityAttribute(DesignerSerializationVisibility.Visible)]
[BrowsableAttribute(true)]
[DslParameter(Name = "assignee")]
[Category("Parameters")]
public string Assignee
{
    get
    {
        return ((string)(base.GetValue(AssigneeProperty)));
    }
    set
    {
        base.SetValue(AssigneeProperty, value);
    }
}

// Other properties - mappings to the rest method parameters

In this way development of DSL activities is simple and straightforward. Now let's look into the base class to see how it's achieved. All the magic is done inside the constructor:

// Initializes parameters by reflecting the typeof(T) members
// and looking for DslParameterAttribute attribute.
private static readonly IList<ParameterInfo> _parameters = GetParameters(typeof(T));

public DslActivity(Type interfaceType, string methodName)
{
    // Set up the interface and method
    InterfaceType = interfaceType;
    MethodName = methodName;

    for (int i = 0; i < _parameters.Count; i++)
    {
        ParameterInfo pi = _parameters[i];
        // This is the main 'trick' - create a binding to 'this' Activity
        // with path set to the relevant property name
        ActivityBind bind = new ActivityBind("/Self", pi.Path);
        WorkflowParameterBinding binding = new WorkflowParameterBinding(pi.ParameterName);
        binding.SetBinding(WorkflowParameterBinding.ValueProperty, bind);
        ParameterBindings.Add(binding);
    }
}

That's all, hope you enjoyed! Download here the sample code for this article.

Domain Specific Language (DSL) with Workflow

Workflow provides two basic activities for interaction with outer world: CallExternalMethodActivity and HandleExternalEventActivity. But sometimes we want to limit our workflow to not use those, since they are too "generic". Instead, we want to provide our set of Domain Specific activities.

The simplest way to achieve this goal is by derivation, providing the required specialization.

Here is a link to a simple solution demonstrating the technique. In this example I create 2 application tasks and wait for completion of any of them. All the work is done with custom activities, defining my DSL.

Workflow (WF) RuleEngine without Workflow

Rule engine is a very useful feature and not always we want the entire Workflow to get into play.

Here is a simple code snippet showing how one can use RuleCondition/RuleSet outside the Workflow scope:

WorkflowMarkupSerializer serializer = new WorkflowMarkupSerializer();
using (Stream s = typeof(Program).Assembly.GetManifestResourceStream("SimplePolicyWorkflow.rules"))
{
    RuleDefinitions rd = (RuleDefinitions)serializer.Deserialize(XmlReader.Create(s));
    RuleCondition c = rd.Conditions[0]; // or RuleSet

    //typeof(rule context) should be passed
    RuleValidation v = new RuleValidation(typeof(Program), null);

    if (c.Validate(v))
    {
        //rule context instance should be passed
        RuleExecution exe = new RuleExecution(v, new Program());
        c.Evaluate(exe);
    }
}

Self Code Review Methodology

When I worked as a team leader I didn't want any code to get into the product without the code review in addition to the unit tests added. But good code review takes time, so I started thinking how I can optimize the process...

First, I came to a conclusion that a review actually consists of two parts:

• Code logic. This is best done by challenging. If the developer quickly and correctly answers to questions, he probably thought about that scenario and covered it. If not, we open the relevant code and check its logic in depth. This gives the opportunity to concentrate on less 'polished' code, analyze it deeper and correct more issues.
• Code sanity, which includes:
  • Code format - ensured by IDE.
  • Code style - which actually is a routine check list. Bingo! Why not have this check list written and given to the developers to do themselves to save the reviewer's time?

Below is the check list I created, please leave your comments...

For each added field

1. Consider type safety. I.e. when the field is used, no cast is required.
2. Ensure its access modifier is private.
3. Ensure its value cannot be computed by other means (using other fields/methods). If yes, it means this field is for caching purposes or used by some setter. If it's for caching:
   • Ensure this caching is required and comment this in code.
   • Ensure it is always in sync with computed value and comment how this is achieved in code.
4. Consider marking this field 'readonly'. If not possible, consider refactoring resulting this field becomes 'readonly'.

For each added property

1. Ensure it has a getter.
2. Consider removing a setter.
3. Ensure set/get parity, i.e. if some value is set, the same one is get.
4. Ensure sequential gets return a same logical value.

For each added property/method

1. Consider minimizing its access scope (private static <--> public virtual)
2. Ensure all the arguments are checked and relevant exception are thrown.

For each added class

1. Consider minimizing its access scope (private sealed inner class <--> public not sealed).
2. Ensure it has a minimal set of constructors.
3. Consider refactoring to have one constructor with initialization logic and other forwarding to it. If needed, add a private constructor.
4. If there is an override for Equals/Hashcode methods, ensure they come in pair and both are computed from the same data.

For each local variable

1. Consider type safety. I.e. when the variable is used, no cast is required.
2. Minimize variable scope.
3. Ensure proper clean up in finally blocks; usually initailization should be right before the try block and not inside it.

For each added line of code

1. Consider implicit impacts (boxing, objects creation, computation).
2. Ensure that every System.SystemException derived exception can be possibly thrown is by intention. (NullReference, Cast etc).

For each 'lock' statement

1. Consider using a framework class, which has the required synchronization built-in. For example, if you need a synchronized Hashtable, don't do locking yourself, but create it with Hashtable.Synchronized() / SynchronizedKeyedCollection<K, T>.
2. Otherwise ensure the design is discussed with your manager.

MVC Using Workflow (WF) as Model/Controller and ASP.NET as View

Download source code for this article

Agenda

Since workflow aims to provide an application business logic engine, it seems natural that we should be able to plug to it a presentation technology of our choice.

As usual, let's start from the requirements to our solution:

1. Total separation between View and Workflow (Model/Controller) logic. We assume that Workflow is provided by an external team.
2. Workflow should be developed with no assumptions on the presentation technology used.
3. The components should be plugged by configuration.

Workflow definition

We assume it is an event driven state machine. This is how it looks for our example:

For each State we will create an interface with:

1. Initialize() method, which is responsible for state initialization (in our case opening a form to collect data from the user).

2. Several events that can happen in this state (user submits data).

3. SubmitXXX methods, each of them raising a corresponding event.

For example, for a State where we need to collect a customer name we will create a following interface:

[ExternalDataExchange] //required for Workflow binding 
public interface IGetNameState
{
   event EventHandler NameReceived;

   void Initialize(Guid instanceId);
   void SubmitName(Guid instanceId, string first, string last); //raises NameReceived event. 
} 

At the end of the StateInitializationActivity we invoke the corresponding Initialize() method:

to show the appropriate view.

Once we are done for each State, we can provide our workflow to the Presentation team.

Presentation

First we must admit that the controller provided by the Workflow does not show any form. Further, it cannot do it by definition, since it does not know how! Instead it calls a relevant Initialize() method to be implemented by the concrete Controller.


That means that we must provide a concrete Controller implementation to perform the actual navigation, i.e:

void IGetNameState.Initialize(Guid instanceId)
{
   HttpContext.Current.Server.Execute
       ("~/GetName.aspx?InstanceId=" + instanceId, false);
}

Later on we will see how we plug this Controller in by configuration.

Now for each state the correct form is shown to the user, but his actions are not handled. Let's do it:

protected void Button1_Click(object sender, EventArgs e)
{
   WorkflowRuntime workflowRuntime = (WorkflowRuntime)Context.Application["WorkflowRuntime"];

   workflowRuntime.SubmitName(
       new Guid(Request.QueryString["InstanceId"]),
       txtFirstName.Text, txtLastName.Text);

   // Once we call Workflow, it's responsible to route the request 
   Response.End();
} 

Initialization and Configuration

• Initialization:
  
  In the Application_Start handler we initialize the WorkflowRuntime:
  
  

  WorkflowRuntime workflowRuntime = new WorkflowRuntime("WorkflowRuntime"); //passing the configuration section name 
  
  workflowRuntime.StartRuntime();
  
  Application["WorkflowRuntime"] = workflowRuntime;

  and in Application_End we stop it:

  WorkflowRuntime workflowRuntime = (WorkflowRuntime)Application["WorkflowRuntime"];
  workflowRuntime.StopRuntime();

• Configuration:
  
  Let's configure ExternalDataExchangeWorkflowService with our concrete Controller type:
  

  <ExternalDataExchangeWorkflowServices>
     <Services>
         <!-- The concrete Controller implementation -->
         <add type="Workflow.RuntimeServices.Controller, Controller, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3a221384c77ffbe5"/>
     </Services>
  </ExternalDataExchangeWorkflowServices>
  <WorkflowRuntime>
     <Services>
         <add type="System.Workflow.Runtime.Hosting.ManualWorkflowSchedulerService, System.Workflow.Runtime, Version=3.0.00000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
         <add type="System.Workflow.Activities.ExternalDataExchangeService, System.Workflow.Activities, Version=3.0.00000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" ConfigurationSection="ExternalDataExchangeWorkflowServices"/>
     </Services>
  </WorkflowRuntime>

  Summary

  This tutorial helps to build a skeleton implementation of ASP.NET front-end for Workflow Controller. In our design we put a special attention on separation and correct interaction between the two. A very important property is that the Workflow does not dictate the implementation to the Presentation layer, its work can be performed in one or several steps. Only when all the information required by the Workflow is collected, it is invoked. For example, to collect the user full name, consisting of first and last names, the presentation may open one or two forms; provide any validation logic it wants and only at the end Submit the information. Such approach separates the Big application flow from the Presentation, opening a possibility to customize/update/change technology/upgrade it in any possible way.