Remoting Probe

• Download source files - 227 Kb

Contents

• Introduction
• Probe internals
  • Concept and design
  • Position probe
  • Configuring probe
  • Publisher
  • Checkpoints
  • Client-Activated Object
  • Installation
• Usage
  • Subscribers
• Remoting Analyzer Studio
  • Analyzing first sample
• Conclusion

Introduction

Consuming remote objects in the distributed .NET applications is fully transparent. This great feature of the .NET technology allows an easy deployment and mapping the logical business model to its physical environment. Behind all this "hidden work" is the remoting paradigm. When the client makes the remoting call, the remoting infrastructure packs all information into the message IMethodCallMessage and pass it through the communication channel to the server side, where its dispatcher unpacks the message to obtain the information for invoking a method on the remote object. The result of the remoting call is going back to the caller in the same manner using the message IMethodReturnMessage. The remoting messages travel through the channel sink stack located on both ends - client and sever. Using the chaining sink parading concept makes it easy to plug-in a custom sink to publish the body of the remoting messages.

Subscribing to the remote probes, we can obtain a knowledge base of the business workflows, which can be used for their post-processing tasks such as monitoring, analyzing, modeling, simulation, tuning, error catch tracer (like an aircraft black box) etc.

This article describes how to design and implement this custom sink called as remoting probe. Also I will give you a lite tool - Remoting Analyzer Studio to see how to subscribe and use the remoting probe notifications and business workflows between the remote objects and their consumers. For instance, you can catch the consumer call - IMethodCallMessage and then simulate it many times from the Remoting Analyzer. It is a useful .NET remoting tool for any stage of the development and production environment.

Let's look in-depth of the remoting probe. I am assuming that you have a knowledge of the .NET remoting infrastructure.

Probe internals

Concept and design

The concept of the loosely coupled message notifications is based on using the COM+ Event Publisher/Subscriber design pattern, where the event call is fired in the proper place of the message sink. The following picture shows that:

There are two places to publish the messages in the probe. As the picture shows, the first one is a "call" checkpoint to publish the IMethodCallMessage message. The other one is when the remote call is returned back from the remote object - checkpoint "return" or "error" on the IMethodReturnMessage message. That's all about the checkpoints in the probe, it was easy to determinate them. More thinking time has to be concentrated on the abstract definition of the event call, such as what kind of the info to publish and how to suitably bind it without loosing the message performance through.

The solution is to divide the published info into the following knowledge bases:

• Source - This is a static information constructed during the remoting provider sink registration process such as process name, process id, probe id, channel type, etc. This information is called as source and its format is name=value in the string text. From the performance point of view, the best way is to use the constructed string in prior and then pass it to the subscriber to the post-processing parsing.
• Remoting - This is a dynamic information related to the remoting mechanism such as message type, id, etc. Also, this info is formatted into the string.
• Business - This is a dynamic information related to the application such as method name, arguments, types, values, etc. It's a bulk of details packed by the IMessage envelope and formatted into the byte array or XML text (binary or SOAP formatter). Note that using the SOAP formatter, the performance is dropped down comparing to the binary one.

Based on the above definitions, the event meta class has been designed as shown in the following snippet:

#region interfaces
[Guid("43603CAC-1D53-4cb0-A3A3-147B2B27A6EF")]
public interface IRemotingProbe
{
   void WriteLogMessage(
               DateTime timestamp, //timestamp of the sample 

               string strSource,   //sourcer

               string strMsgType,  //type

               string strMsgId,    //identifier

               object objMsgBody); //message body

               }
#endregion


#region classes
[Guid("768DFCE9-279E-4818-8964-FA6D368BAC78")]
[EventClass]
[Transaction(TransactionOption.NotSupported)] 
[ObjectPooling(Enabled=true, MinPoolSize=25, MaxPoolSize=100)]
[EventTrackingEnabled]
public class RemotingEventClass : ServicedComponent, IRemotingProbe
{
   public void WriteLogMessage(
              DateTime timestamp, //timestamp of the sample 

              string strSource,   //sourcer

              string strMsgType,  //type

              string strMsgId,    //identifier

              object objMsgBody   //message body

              )
   {
      throw new Exception("This class can be called 
                       only by the COM+ Event System");
   } 
}
#endregion

where:

• timestamp is the checkpoint DateTime
• strSource is the identifier of the probe, host process, channel, etc.
• MsgType is the type of the event call. There are the following types:

  public class ProbeType 
  { 
      // publish creating
  
      public const string create = "CREATE"; 
      // publish pre-processing
  
      public const string precall = "CALL";  
      // publish one way pre-processing
  
      public const string oneway = "ONEWAY";
      // publish post-processing 
  
      public const string postcall = "RETURN";  
      // publish closing
  
      public const string close = "CLOSE"; 
      // publish error
  
      public const string error = "ERROR"; 
      // publish ping status
  
      public const string ping = "PING";
  }

• strMsgId is the Guid of the session call. Each call has own unique guid generated by the probe and used for all messages related with this call. It's traveling in the message property bag.
• objMsgBody represents the message body depending on the type of the message and message formatter.

The remoting probe fires the event call in the following situations:

• the probe has been registered - CREATE
• the probe has been unregistered (or host process is going to shutdown) - CLOSE
• the probe is pinging. Each setup time the probe fires the event to notify about its state - PING
• the Message is passing through the call checkpoint - CALL or ONEWAY
• the Message is passing through the return checkpoint - RETURN or ERROR

The probe's checkpoints are controllable via the config file - statically or programmatically using the remoting LogicalCallContect design pattern mechanism.

The probe is listening for the following call Context object:

[Serializable]
public class ProbeLogicalCallContext : ILogicalThreadAffinative
{
   string m_strKey;        //probe key

   string m_strCtrl;       //control

   string m_strStatus;     //status 

   public string Key 
   { 
       get {return m_strKey; } 
       set { m_strKey = value; }
   }
   public string Ctrl
   { 
       get {return m_strCtrl; } 
       set { m_strCtrl = value; }
   }
   public string Status
   { 
       get {return m_strStatus; } 
       set { m_strStatus = value; }
   }
}

The ProbeLogicalCallContext is broadcasting over the remoting to scan the probes in the registered channels to obtain their ID and status or send to the specified probe using the probeId (Key). When the specified probe is served or the current probe located at the endpoint, the exception is thrown with the probe's status info. This is a loosely coupled design pattern to control the probe located anywhere on the remoting network.

Using the above design, the subscriber or any remoting consumer can control the probe behavior such as turn the probe on/off or only its particular checkpoint. The default setup of the probe is off (disabled).

Position probe

The probe is a custom message sink driven by the IMessage interface. It's position can be anywhere before the client or after the server formatter sink (this version doesn't handle the streamed messages). Number of probes in the channel is not limited, so if it's necessary to publish modified messages by other custom sinks, the probe can be plugged-in between them. The position of the probe is taken from the config file, like shown in the following snippet:

<configuration>
 <system.runtime.remoting>
  <application>
   <service>
    <wellknown mode="SingleCall" type="TestRemoteObject.EchoObject, 
                TestRemoteObject, Version=1.0.977.40765,
                Culture=neutral, PublicKeyToken=b2195ac3e95b2650" 
                objectUri="endpointEcho" />
   </service>
   <channels>
    <channel ref="tcp" port="1234" >
     <serverProviders>
      <formatter ref="binary" />
      <provider ref="probe" name="ProbeS1234" 
                call="true" return="true" error="true"  
                binary="true" endpoint="endpointEcho"/>
     </serverProviders> 
    </channel>
   </channels>
  </application>
 </system.runtime.remoting>
</configuration>

Configuring probe

The remoting probe uses the following standard and custom properties:

• ref is the provider template being referenced (for instance: ref="probe")
• name specifies the name of the provider. Each provider has an unique name which is used for the proper message (for instance: name="ProbeS1234")
• call (custom property) specifies the control bit of the call checkpoint, default value is false. When this bit is setup for true, the IMethodCallMessage is published.
• return (custom property) specifies the control bit of the return checkpoint, default value is false. When this bit is setup for true, the IMethodReturnMessage is published.
• error (custom property) specifies the control bit of the error checkpoint, default value is false. When this bit is setup for true, only the IMethodReturnMessage with exception is published.
• binary (custom property) specifies the type of the formatter for event call, default value is true, which means the binary formatter is used, in opposite the SOAP formatter.
• endpoint (custom property) specifies the name of the registered endpoint in the current host process. This property is necessary for .NET SP2 +, which it not allows to pass IMessage through the binary formatter without the registered endpoint. This property is used only by the server probe.
• pingtime (custom property) specifies the period of the pinging in seconds. The default value is minimum 30 seconds. The value 0 will turn off the pinging feature. Note that this property is not dynamically controllable and it is implemented only in the server probe.

Publisher

The publisher is located in the probe. It's a light function to invoke the event call in the fire & forget manner using the delegate design pattern like shown in the following code snippet:

public void Write(string strSource, string strMsgType, 
                                     string strMsgId, object msg) 
{
   try 
   {
      //publishing in manner of the fire & forget

      delegatePublisher delegator = new delegatePublisher(Publisher);
      delegator.BeginInvoke(DateTime.Now, strSource, strMsgType, strMsgId, 
      msg, null, null); 
   }
   catch(Exception ex) 
   {
      Trace.WriteLine(ex.Message);
   }
}

public void Publisher(DateTime objTimeStamp, string strSource, 
                      string strMsgType, string strMsgId, object msg) 
{
   try 
   {
      //publishing

      using(RemotingEventClass pec = new RemotingEventClass()) 
      {
         pec.WriteLogMessage(objTimeStamp, 
            strSource, strMsgType, strMsgId, msg);
      }
   }
   catch(Exception ex) 
   {
      Trace.WriteLine(ex.Message);
   }
}

The probe logic packs required arguments in the specified format and then calls the publisher wrapper function to perform fire & forget event call. After that, the call is back to the normal message flow which is the forwarding message to the next sink. Note that the publisher doesn't care whether the subscriber included its business logic purpose too, that's the LCE system notification feature. In other words, the publisher's target is the COM+ Event System and this service makes the bridge to the subscriber based on its subscription registered in the event store.

Checkpoints

The probe checkpoints are places where remoting message is strobed and send to the publisher. There are actually two checkpoints: call and return. The return checkpoint can publish any return message including the error message. Internally this checkpoint has another checkpoint to decide that only the error message will be published.

The following code snippet shows the checkpoints in the server probe, when ProcessMessage function is invoked:

public ServerProcessing ProcessMessage(IServerChannelSinkStack sinkStack, 
                   IMessage requestMsg, 
                   ITransportHeaders requestHeaders, 
                   Stream requestStream, 
                   out IMessage responseMsg, 
                   out ITransportHeaders responseHeaders, 
                   out Stream responseStream)
{

   ServerProcessing servproc = ServerProcessing.Complete;
   responseMsg = null;
   responseHeaders = null;
   responseStream = null;


   //Are we in the business?

   if(m_Next != null) 
   {
      //workaround

      requestMsg.Properties["__ObjUri"] = 
              requestMsg.Properties["__Uri"];

      //create a checkpoint id if doesn't exit in the IMessage

      if(requestMsg.Properties.Contains(ProbeTicket.checkpoint) == false)
      {
         requestMsg.Properties[ProbeTicket.id] = Guid.NewGuid();
         requestMsg.Properties[ProbeTicket.checkpoint] = (int)0; 
      }

      //pre-processing probe (call)

      m_Probe.Write(m_Provider.m_ProbeState, requestMsg, null); 

      //mark the checkpoint

      if(requestMsg.Properties[ProbeTicket.checkpoint] is int) 
      {
         int cp = (int)requestMsg.Properties[ProbeTicket.checkpoint]; 
         requestMsg.Properties[ProbeTicket.checkpoint] = ++cp;
      }

      //check the Probe Call Context for this probe and change its state 

      string result = UpdateProbeState(requestMsg);
      if(result == null) 
      {
         //processing message in the current channel 

         servproc = m_Next.ProcessMessage(sinkStack, 
                     requestMsg, requestHeaders, 
                     requestStream, out responseMsg, 
                     out responseHeaders, out responseStream);
      }
      else 
      {
         //this messages has been dedicated for probe only

         RemotingException rex = new 
             RemotingException(ProbeCallContext.exceptionMsg);
         rex.Source = result;
         responseMsg = new ReturnMessage(rex, 
             (IMethodCallMessage)requestMsg);
         responseHeaders = requestHeaders;
      }

      //check our properties 

      if(responseMsg != null) 
      {
         if(responseMsg.Properties.Contains(ProbeTicket.checkpoint) 
                                                           == false)
         {
            responseMsg.Properties[ProbeTicket.id]=
                  requestMsg.Properties[ProbeTicket.id];
            responseMsg.Properties[ProbeTicket.checkpoint] = 
                  requestMsg.Properties[ProbeTicket.checkpoint]; 
         }

         //mark the checkpoint

         if(responseMsg.Properties[ProbeTicket.checkpoint] is int) 
         {
            int cp = (int)responseMsg.Properties[ProbeTicket.checkpoint]; 
            responseMsg.Properties[ProbeTicket.checkpoint] = --cp;
         }

         //post-processing probe (return or error)

         m_Probe.Write(m_Provider.m_ProbeState, responseMsg, requestMsg); 
      }
   } 
   else 
   {
      //We have no active sink

      Trace.WriteLine
        (string.Format("{0}:RouterServerSink ProcessMessage null", 
        m_Provider.ProviderName));
   }
   return servproc;
}

Client-Activated Object

The remoting infrastructure is based on the IMethodCallMessage and IMethodReturnMessage. These two messages are published by the probe. However, there is a special case related to the Client-Activated Object (CAO) mechanism. The remote object is activated by the client using the Activator.CreateInstance (..) method with the arguments such as object type, construction arguments and URL. This is a wrapper (helper) function using the standard remoting message to call a RemoteActivationService endpoint to perform a construction of your endpoint object. This MethodCall message has only one argument passing the client info. Its type is an IConstructionCallMessage and it's derived from the IMethodCallMessage interface. The probe is monitoring this situation and publishing this message instead of the invoked call message. See the following code snippet:

//check if we running the client-activated object

if(mcm.Args != null && mcm.ArgCount == 1 && mcm.Args[0] 
                              is IConstructionCallMessage) 
{
   //activator argument

   msg = mcm.Args[0] as IConstructionCallMessage;
   //workaround

   msg.Properties["__ObjUri"] = mcm.Properties["__ObjUri"];
   msg.Properties[ProbeTicket.id] = mcm.Properties[ProbeTicket.id];
   msg.Properties[ProbeTicket.checkpoint] = 
                mcm.Properties[ProbeTicket.checkpoint];
}

The same situation is done for a message back. The server activator returns its result in the return value of the IMethodReturnMessage as an object with the type IConstructionReturnMesssage.

Note that there is a small workaround when the server activator throws exception. This message can't be serialized for its publishing using the binary formatter, so the SOAP formatter is automatically used to publish this error message.

Installation

The remoting probe solution consists of the following projects:

• MessageProbe.dll - This is an assembly of the server and client custom providers.
• RemotingProbeEventClass.dll - This is an assembly of the event meta class and interfaces.

Installation steps:

1. Install the MessageProbe assembly into the GAC
2. Modify the machine.config file, section of the ClientProviders inserting the remoting probe

   <provider id="probe" type="RKiss.MessageProbe.ProbeClientSinkProvider, 
                      MessageProbe, Version=1.0.959.33740, 
                      Culture=neutral, PublicKeyToken=2a13ad995670cdcf"/>

3. Modify the machine.config file, section of the ServerProviders inserting the remoting probe

   <provider id="probe" type="RKiss.MessageProbe.ProbeServerSinkProvider, 
                    MessageProbe, Version=1.0.959.33740, 
                    Culture=neutral, PublicKeyToken=2a13ad995670cdcf"/>

4. Register the RemotingProbeEventClass assembly as an Enterprise Service.

Also, you can use the InstallIntoTheGAC.bat file included in this solution to complete steps 1 and 4.

After the above steps, the remoting probe is ready to use in the remoting config files.

Plugging in the probe into the provider can be done with the default setup like shown:

<provider ref="probe" name="ProbeS1234" />

In this config case, the probe will monitor the host process firing the CREATE and CLOSE events and the server probe also will perform firing the PING event every ping time.

This is a minimum publishing notification from the remoting probes. They can be ran without any impact for the remoting performance.

Usage

The remoting probe usage is dependent on its subscriber. The probe without using any subscriber is a passive sink, publishing the messages going to nowhere. The usage of the remoting probe has two phases:

• Incorporating the remoting probes in the remoting config files during the deployment process. This task is very simply, but important. It gave the opportunity to build incrementally, more sophisticated application models.
• Build the proper subscriber to consume the knowledge base published by the probes.

Subscribers

There can be many number of different subscribers designed and implemented for the remoting probes. From the simplest one, like the host process monitoring, to more complicated business control loops based on the strobed business knowledge base. Some solutions can be lead to more sophisticated model than the main application.

Basically, the subscribers are divided into the following two groups:

• passive, where a received information is used for post-processing such as monitoring, analyzing, modeling, simulation and another off-line tools.
• active, where a received information is used to control business model - its workflow such as tuning, balancing and etc. - on-line solutions.

How powerful is the remoting details published by the probe, is shown in my included solution called Remoting Analyzer Studio.

Here are its details:

Remoting Analyzer Studio

The Remoting Analyzer Studio is the GUI Subscriber to the remoting probe(s) to explore the remoting messages between the remote objects and their consumers. It has been designed and implemented (using "as it is" methodology) to demonstrate power of the published knowledge base by remoting probe. It can be used for developing, testing and tracing the remoting issues during the product lifecycle. Note that the Analyzer is a full isolated - loosely coupled design pattern from your remoting objects and consumers.

The concept of the Remoting Analyzer Studio is based on exploring the incoming event calls into the TreeView layout. Basically, there are two kinds of panels with the splitter. The left panel is dedicated to the probes and simulation of the remoting call. The second one, on the right side is for storing the incoming samples - event calls into the TreeView for the post-processing examination.

The following screenshot shows the Probes tab, where each accessible probe has its own entry. It's the read-only TreeView layout except the ProbeStatus node, which can be edited to control the probe behavior. The probe entry is done automatically either by the pinging or scanning probes.

The next tab is for scanning the remoting probes. You can create your entry of the remoting URI address and click for its scanning. This TreeView is persisted into the RemotinfAnalyzerStudio.xml file to automatically scan the probes during the Analyzer start-up process.

The Simulation tab has capability to drop the MethodCall node from the samples TreeView (left panel) and invoking this method again from the Remoting Analyzer Studio. Simulation is a powerful feature when the caught error on your server object can be simulated again in the post-processing manner. Note that there are some limitations, in this version you can't change any value of the arguments or simulate a client-activated object.

The following code snippet shows the implementation of invoking the IMethodCallMessage message in the post-processing manner. The trick is to find the proper channel and forward the message to its first sink:

private void menuItemSimulatorRun_Click(object sender, System.EventArgs e)
{
   TreeNode tnSel = treeViewSimulator.SelectedNode;
   IMethodReturnMessage responseMsg = null;
   try
   {
      if(tnSel.Tag is IMethodCallMessage) 
      {
         IMethodCallMessage requestMsg = tnSel.Tag as IMethodCallMessage;
         string strDummy = null;
         IMessageSink iMsgSink = null;

         string strObjUrl = "?";
         foreach(TreeNode tnUri in tnSel.Nodes) 
         {
            if(tnUri.Tag is string) 
            {
               strObjUrl = tnUri.Tag as string; 
            }
         }

         //create a checkpoint id in the IMessage

         requestMsg.Properties[ProbeTicket.id] = Guid.NewGuid();
         requestMsg.Properties[ProbeTicket.checkpoint] = (int)1000; 

         //find the properly outgoing channel registered in this process

         foreach(IChannel channel in ChannelServices.RegisteredChannels)
         {
            if(channel is IChannelSender)
            {
               iMsgSink = (channel as IChannelSender).CreateMessageSink
                                        (strObjUrl, null, out strDummy);
               if(iMsgSink != null) 
                  break; 
            }
         }
         if(iMsgSink == null)
         {
            statusBarPanelResponse.Text = "There is no properly 
                                 channel to invoke the remote method";
         }
         else 
         {
            //Pass the IMessage to the following channel 

            //based on the method's attribute

            //The SyncProcessMessage can not be done on 

            //the OneWay attributed method (deadlock process)

            if(RemotingServices.IsOneWay(requestMsg.MethodBase) == true) 
            {
               iMsgSink.AsyncProcessMessage(requestMsg, null);
            }
            else 
            {
               responseMsg = (IMethodReturnMessage)
                              iMsgSink.SyncProcessMessage(requestMsg);
            }

            if(responseMsg != null && responseMsg.Exception == null) 
            {
               statusBarPanelResponse.Text = "Done";
            }
            else
            if(responseMsg != null && responseMsg.Exception != null) 
            {
               throw responseMsg.Exception;
            }
         }
      }
   }
   catch(Exception ex) 
   {
      statusBarPanelResponse.Text = "Error: " + ex.Message;
   }
}

Analyzing first sample

I built the test solution to demonstrate capability of the Remoting Analyzer Studio. You can find it in a separate folder \Test\bin:

• TestRemoteObject.dll - remote SAO object
• TestHostServer.exe - host server process
• TestWindowsClient.exe - consumer of the remote object

Note that the test requires to install the assembly TestRemoteObject into the GAC in prior. The probe and Analyzer has been implemented using .NET SP2 and tested on the Windows 2000 Advanced Server. It should work without any problem also on XP machines.

The following picture shows the test schema of the components and programs:

The Analyzer can be launched to the test environment any time. Subscribing the Analyzer to the event system will start its scanner routine to find the probes on the specified remoting addresses (see Scanning tab). Also, the probe pinging will notify existence of the probe in the remoting channels (see Probes tab).

Echo button

The client invokes the remote method Echo with many different types of arguments including a Dataset. When you click the button Echo on the TestClient panel, the remoting method is called on the remote object using the the IMethodCallMessage and IMethodReturnMessage messages. These messages are strobed by the probes located at the ends (client and server) and they are published to the Event System. Because, the Analyzer has been subscribed, these samples are received and explored on the right panel of the Analyzer. Now we can examine the remoting samples (messages) between the remote object and its consumer. Note that the samples are not received in order how they have been published, because the Probe publisher is using the delegate fire & forget design pattern. Each sample has its own unique identifier and using the next feature - tracer, it is easy to walk through them in the order how they passed a specified checkpoint.

F5 Key (Tracer)

Selecting the sample on the Analyzer and pressing the key F5, all samples related for this remoting session will be expanded and ready to walk through them:

As you can see the above screenshot, the Analyzer received the remoting and business knowledge base for this call. For instance: Dataset, Tables, Rows, Types, Columns, etc. Pressing the F5 Key you step to the next message workflow. This stepping is circulated and for its exit, use the Ctrl + F5 or just select another session's sample.

Drag & Drop MethodCall

The MethodCall (only for SAO) can be dragged and dropped into the left panel (Simulation tab) to make its post-processing simulation call. The probe and server object will response this call in the same manner like its originator (in this case the Echo button). Let me assume that you catch the error for wrong value of the Dataset record. Based on this feature you can duplicate the method call and figure out the problem.

F9 Key (Mark)

Pressing the F9 key, the sample is marked and the mouse tool tip shows some important info about the pointed sample:

To exit this mode, just press the Ctrl + F9 or click on the MarkExit menu. This feature also calculates a delta time from the marked sample message, so it is easy to see a timing between the MethodCall and MethodResponse messages.

ON/OFF switcher

The Remoting Analyzer can be subscribed/unsubscribed to/from the Event System using this button. It's a good practice to receive the required remoting samples and then turn off the Analyzer to work off-line. (analyzing the samples).

What next ...

Now, configure your remoting project with the probe and make the same process described above. Note that the Analyzer is type driven, so your assembly has to be accessible locally or through the GAC. The other choice is to use the SOAP formatter in the probe.

Another note - is about the enhancing feature of the Analyzer. I am planning to add a capability of triggering, filtering and circulating samples, full simulation of the MethodCall, remoting subscribing, and more. These new features will be allowed to keep the subscriber run in the background, as a history of the remote object consuming for the post-processing investigation, tuning, simulation and animation of the business workflow.

Conclusion

Using the LCE remoting probes in the Distributed Application Framework to strobe the business workflow gives you an additional post-processing knowledge base which can be used for many things. This is an incrementally development pyramid based on developing the suitable subscribers. In this article, I presented one of them - Remoting Analyzer Studio, which is a small tool to examine the contents of the remoting messages.