Using a MessageFilter to avoid IDE threading and timing issues.

Due to timing and threading issues with interacting with the IDE programatically, several errors may occur at seemingly random intervals.  These errors include “Application is busy”, “Callee was rejected by caller”, “The operation could not be completed”, or RPC_E_SERVER…. Exceptions.  Because these are ultimately timing issues the best solution is to use a Message Filter to tell the host to wait and try again when a call is rejected.  In the ShapeAppAdvancedCSharp and TestCon samples a simple MessageFilter is implemented which is included below in CSharp and VB.Net.  To implement this class, be sure to update your main method to use the message filter.  For an alternate implementation check out this post: http://msdn2.microsoft.com/en-us/library/ms228772(VS.80).aspx

Visual Basic.Net-
Update main method:

Shared Sub Main()
   Using messageFilter_Value As New MessageFilter()
      'original contents are unchanged
   End Using
End Sub

Message Filter:

Imports System
Imports system.Runtime.InteropServices

' There are several threading issues in the IDE that can cause the STA to get hijacked while
' we're attempting to automate. By default, .Net does not provide a message filter, so
' blocked calls result in RPC_E_SERVER... exceptions. Since this is a timing issue, it would
' be very hard for the host to trap these exceptions and try again. The easiest solution,
' is to implement a message filter which tells the host to wait and try again
' when a call is rejected.
<ComImport()> _
<Guid("00000016-0000-0000-C000-000000000046")> _
<InterfaceType(ComInterfaceType.InterfaceIsIUnknown)> _
Public Interface IMessageFilter
   <PreserveSig()> _
   Function HandleInComingCall(ByVal dwCallType As Integer, _
      ByVal hTaskCaller As IntPtr, _
      ByVal dwTickCount As Integer, _
      ByVal lpInterfaceInfo As IntPtr) As Integer

   <PreserveSig()> _
   Function ReTryRejectedCall(ByVal hTaskCallee As IntPtr, _
      ByVal dwTickCount As Integer, _
      ByVal dwRejectType As Integer) As Integer

   <PreserveSig()> _
   Function MessagePending(ByVal hTaskCallee As IntPtr, _
      ByVal dwTickCount As Integer, _
      ByVal dwPendingType As Integer) As Integer
End Interface

' This needs to be MBRO because winforms will query for this to obtain the component manager
' When it does this, it will call Marshal.GetObjectForIUnknown which will convert this to
' an object and force serialization to type MessageFilter. To avoid this, make it MBRO
' so the cast to com visible types will succeed.
Friend Class MessageFilter
   Inherits MarshalByRefObject
   Implements IDisposable
   Implements IMessageFilter

   Private disposedValue As Boolean = False ' To detect redundant calls
   Private disposed As Boolean
   Private oldFilter As IMessageFilter
   Private Const SERVERCALL_ISHANDLED As Integer = 0
   Private Const PENDINGMSG_WAITNOPROCESS As Integer = 2

   'IDisposable
   Protected Overridable Sub Dispose(ByVal disposing As Boolean)
      If Not Me.disposed Then
         Dim dummy As IMessageFilter
         Dim hr As Integer

         hr = MessageFilter.CoRegisterMessageFilter(Me.oldFilter, dummy)
         System.Diagnostics.Debug.Assert(hr >= 0)
         Me.disposed = True
         System.GC.SuppressFinalize(Me)
      End If
   End Sub

   #Region " IDisposable Support "
   ' This code added by Visual Basic to correctly implement the disposable pattern.
   Public Sub Dispose() Implements IDisposable.Dispose
      ' Do not change this code. Put cleanup code in Dispose(ByVal disposing As Boolean) above.
      Dispose(True)
      GC.SuppressFinalize(Me)
   End Sub
   #End Region

   Public Function HandleInComingCall(ByVal dwCallType As Integer,
                                      ByVal threadIdCaller As System.IntPtr,
                                      ByVal dwTickCount As Integer, ByVal lpInterfaceInfo As System.IntPtr)
                                       As Integer Implements IMessageFilter.HandleInComingCall
      'Return the ole default (don't let the call through)
      Return MessageFilter.SERVERCALL_ISHANDLED
   End Function

   Public Function MessagePending(ByVal threadIdCallee As System.IntPtr,
                                  ByVal dwTickCount As Integer,
                                  ByVal dwPendingType As Integer)
                                  As Integer Implements  IMessageFilter.MessagePending
      'perform default processing
      Return MessageFilter.PENDINGMSG_WAITNOPROCESS
   End Function

   Public Function ReTryRejectedCall(ByVal threadIdCallee As System.IntPtr,
                                     ByVal dwTickCount As Integer,
                                     ByVal dwRejectType As Integer)
                                   As Integer Implements IMessageFilter.ReTryRejectedCall
      ' Retry the call in 100 milliseconds
      ' Normally, a host would eventually want to throw a message box after some pre-determined time-out if
      ' the call doesn't get through (like excel does after a few minutes).
      ' "Microsoft Excel is waiting for an ole call to complete"
      ' I leave this as an excecise for the reader.
      Return 100
   End Function

   Friend Sub New()
       Dim hr As Integer

       Me.disposed = False
       hr = MessageFilter.CoRegisterMessageFilter(CType(Me, IMessageFilter), Me.oldFilter)
       System.Diagnostics.Debug.Assert(hr >= 0)
    End Sub

   <DllImport("ole32.dll")> _
   <PreserveSig()> _
   Private Shared Function CoRegisterMessageFilter(ByVal lpMessageFilter As IMessageFilter,
                                                   ByRef lplpMessageFilter As IMessageFilter) As Integer
   End Function
End Class

CSharp-
Update main method:

static void Main()
{
    using (MessageFilter messageFilter = new MessageFilter())
   {
       // original contents are unchanged
   }
}

Message Filter:

//***************************************************************************
//
// Copyright (c) 2006 Microsoft Corporation. All rights reserved.
// This code is licensed under the Visual Studio SDK license terms.
// THIS CODE IS PROVIDED "AS IS" WITHOUT WARRANTY OF
// ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY
// IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR
// PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.
//
//***************************************************************************
using System;
using System.Runtime.InteropServices;

// There are several threading issues in the IDE that can cause the STA to get hijacked while
// we're attempting to automate. By default, .Net does not provide a message filter, so
// blocked calls result in RPC_E_SERVER... exceptions. Since this is a timing issue, it would
// be very hard for the host to trap these exceptions and try again. The easiest solution,
// is to implement a message filter which tells the host to wait and try again
// when a call is rejected.
namespace Microsoft.VisualStudio.Tools.Applications.Samples.ShapeApp
{
   [ComImport()]
   [Guid("00000016-0000-0000-C000-000000000046")]
   [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
    public interface IMessageFilter
   {
      [PreserveSig]
      int HandleInComingCall(
         int dwCallType,
         IntPtr hTaskCaller,
          int dwTickCount,
         IntPtr lpInterfaceInfo);

      [PreserveSig]
      int RetryRejectedCall(
         IntPtr hTaskCallee,
         int dwTickCount,
         int dwRejectType);

       [PreserveSig]
      int MessagePending(
         IntPtr hTaskCallee,
         int dwTickCount,
         int dwPendingType);
   }

    // This needs to be MBRO because winforms will query for this to obtain the component manager
    // When it does this, it will call Marshal.GetObjectForIUnknown which will convert this to
    // an object and force serialization to type MessageFilter. To avoid this, make it MBRO
    // so the cast to com visible types will succeed.
    internal class MessageFilter : MarshalByRefObject, IDisposable, IMessageFilter
    {
       #region Constructors / Destructor / Dispose
       internal MessageFilter()
       {
          this.disposed = false;
          int hr = MessageFilter.CoRegisterMessageFilter((IMessageFilter)this, out this.oldFilter);
          System.Diagnostics.Debug.Assert(hr >= 0);
        }

      ~MessageFilter()
      {
          this.Dispose();
      }

       public void Dispose()
      {
          if (!this.disposed)
         {
            IMessageFilter dummy;
            int hr = MessageFilter.CoRegisterMessageFilter(this.oldFilter, out dummy);
            System.Diagnostics.Debug.Assert(hr >= 0);
            this.disposed = true;
            System.GC.SuppressFinalize(this);
         }
      }

      #endregion

      #region IMessageFilter Implementation
      int IMessageFilter.HandleInComingCall(int dwCallType, IntPtr threadIdCaller, int dwTickCount, IntPtr lpInterfaceInfo)
      {
          // Return the ole default (don't let the call through.
          return MessageFilter.SERVERCALL_ISHANDLED;
      }

      int IMessageFilter.RetryRejectedCall(IntPtr threadIDCallee, int dwTickCount, int dwRejectType)
      {
         // Retry the call in 100 milliseconds
         // Normally, a host would eventually want to throw a message box after some pre-determined time-out if
         // the call doesn't get through (like excel does after a few minutes).
         // "Microsoft Excel is waiting for an ole call to complete"
         // I leave this as an excecise for the reader.
         return 100;
      }

      int IMessageFilter.MessagePending(IntPtr threadIDCallee, int dwTickCount, int dwPendingType)
      {

         // Perform default processing.
         return MessageFilter.PENDINGMSG_WAITNOPROCESS;
       }
       #endregion

       #region Native methods
       [DllImport("ole32.dll")]
       [PreserveSig]
       private static extern int CoRegisterMessageFilter(IMessageFilter lpMessageFilter, out IMessageFilter lplpMessageFilter);
       #endregion

       #region Private fields
       private bool disposed;
       private IMessageFilter oldFilter;
       #endregion

       #region Constants
       private const int SERVERCALL_ISHANDLED = 0;
       private const int PENDINGMSG_WAITNOPROCESS = 2;
       #endregion
   }
}



Posted Aug 07 2007, 01:52 PM by Melody

Comments

Carlos Quintero's blog wrote Frustrations with command-line add-ins for Visual Studio
on 10-11-2007 11:52 AM

Here is another frustration when working with the automation model of Visual Studio: suppose that you

Dhananjay wrote re: Using a MessageFilter to avoid IDE threading and timing issues.
on 12-02-2010 5:43 AM

I have done same changes in my code, I mean implemented same 'IMessageFilter ' interface but still it gives the same error message 'The message filter indicated that the application is busy'.  

I am not sure what is the exactly issue, Can you please help me to resolve the issue?

Here is my code

using System;

using System.Collections.Generic;

using System.Linq;

using System.Web;

using System.Collections;

using Word = Microsoft.Office.Interop.Word;

using System.IO;

using System.Diagnostics;

using System.Configuration;

using System.Runtime.InteropServices;

using System.Threading;

namespace PsLiveDocX.WsHandler

{

   public class AppendDocument

   {

       public AppendDocument()

       {

       }

       string _FileExtension = ".doc";

       [STAThread]

       public AppendWordDocumentResult AppendWordDocument(AppendDocumentRequest requestCriteria)

       {

           AppendWordDocumentResult _result = new AppendWordDocumentResult();

           try

           {

               MessageFilter.Register();

               if (null != requestCriteria && null != requestCriteria.Documents)

               {

                   PsDocument document = new PsDocument();

                   for (int i = 0; i < requestCriteria.Documents.Count; i++ )

                   {

                       document = requestCriteria.Documents[i];

                       if ("DOCX" == document.extension.ToUpper())

                           _FileExtension = ".docx";

                       document.binaryDataInBytes = Convert.FromBase64String(document.binaryData);

                   }

               }

               else

                   throw new Exception("Binary files are required.");

               bool insertPageBreaks = requestCriteria.InsertPageBreaks;

               string _no = DateTime.Now.ToString("s").Replace("-", "").Replace(":", "").Replace("T", "");

               Random randon = new Random();

               _no += randon.Next().ToString();

               string outputFilename = string.Empty;

               string _path = AppDomain.CurrentDomain.BaseDirectory + @"Documents\\";

               if (!System.IO.Directory.Exists(_path))

                   System.IO.Directory.CreateDirectory(_path);

               string _outputFile = "OutputFile_" + _no + _FileExtension;

               outputFilename = _path + _outputFile;

               byte[] _outputFileByteArray;

               if (!AppendWordDoc(requestCriteria.Documents, outputFilename, insertPageBreaks, _path, out _outputFileByteArray, _result))

                   throw new Exception("Error while merging documents.");

               else

               {

                   _result.ResultInfo.IsSuccessful = true;

                   _result.OutputFileByteArray = Convert.ToBase64String(_outputFileByteArray);

               }

           }

           catch (Exception ex)

           {

               MessageFilter.Revoke();

               if (null == _result.ResultInfo)

                   _result.ResultInfo = new ResultStatus();

               if (null == _result.ResultInfo.Messages)

                   _result.ResultInfo.Messages = new List<string>();

               _result.ResultInfo.Messages.Add(ex.Message);

               _result.ResultInfo.IsSuccessful = false;

           }

           MessageFilter.Revoke();

           return _result;

       }

       private bool AppendWordDoc(List<PsDocument> psDocuments, string outputFilename, bool insertPageBreaks, string filePath, out byte[] outputFileByteArray, AppendWordDocumentResult _result)

       {

           List<string> filesToMerge = CreateFilesFromBinnaryData(psDocuments, filePath);

           object _defaultTemplate = filePath + "DefaultTemplate.doc";

           object _missing = System.Type.Missing;

           object _pageBreak = Word.WdBreakType.wdPageBreak;

           object _outputFile = outputFilename;

           object _format = Microsoft.Office.Interop.Word.WdSaveFormat.wdFormatDocumentDefault;

           foreach (System.Diagnostics.Process process in System.Diagnostics.Process.GetProcessesByName("WINWORD"))

           {

               if (process.StartTime.AddMinutes(Convert.ToInt32(System.Configuration.ConfigurationManager.AppSettings["WinWordExeDeleteTime"])) < DateTime.Now)

               {

                   process.Kill();

               }

           }

           // Create  a new Word application

           Word._Application _wordApplication = new Word.Application();

           // Create a new file based on our template

           Word._Document _wordDocument = _wordApplication.Documents.Add(ref _defaultTemplate

                                                                       , ref _missing

                                                                       , ref _missing

                                                                       , ref _missing);

           _result.ResultInfo.Messages.Add("1");

           try

           {

               // Make a Word selection object.

               Word.Selection selection = _wordApplication.Selection;

               // Loop thru each of the Word documents

               foreach (string file in filesToMerge)

               {

                   //_result.ResultInfo.Messages.Add(filePath + file);//

                   // Insert the files to our template

                   selection.InsertFile(filePath + file

                                       , ref _missing

                                       , ref _missing

                                       , ref _missing

                                       , ref _missing);

                   _result.ResultInfo.Messages.Add("2");

                   //Do we want page breaks added after each documents?

                   if (insertPageBreaks)

                   {

                       selection.InsertBreak(ref _pageBreak);

                   }

               }

               _result.ResultInfo.Messages.Add("3");

               // Save the document to it's output file.

               _wordDocument.SaveAs(ref _outputFile

                                   , ref _format

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing

                                   , ref _missing);

               _result.ResultInfo.Messages.Add("4");

               // Clean up!

               _wordDocument.Close(ref _missing, ref _missing, ref _missing);

               _result.ResultInfo.Messages.Add("5");

               FileStream _fls = new FileStream(outputFilename, FileMode.Open, FileAccess.Read);

               byte[] _blobDoc = new byte[_fls.Length];

               _fls.Read(_blobDoc, 0, System.Convert.ToInt32(_fls.Length));

               _fls.Close();

               outputFileByteArray = _blobDoc;

               _result.ResultInfo.Messages.Add("6");

           }

           catch (Exception ex)

           {

               MessageFilter.Revoke();

               _result.ResultInfo.Messages.Add("7");

               _result.ResultInfo.Messages.Add(ex.Message + ex.StackTrace);

               throw ex;

           }

           finally

           {

               _result.ResultInfo.Messages.Add("8");

               // Finally, Close our Word application

               _wordApplication.Quit(ref _missing, ref _missing, ref _missing);

               _wordDocument = null;

               _wordApplication = null;

               System.Threading.Thread.Sleep(500);

               //Delete Temp files

               //foreach (string file in filesToMerge)

               //{

               //    if (File.Exists(filePath + file))

               //        File.Delete(filePath + file);

               //}

               //if (File.Exists(outputFilename))

               //    File.Delete(outputFilename);

               _result.ResultInfo.Messages.Add("9");

           }

           return true;

       }

       private List<string> CreateFilesFromBinnaryData(List<PsDocument> psDocuments, string filePath)

       {

           List<string> _filesToMerge = new List<string>();

           try

           {

               if (psDocuments != null && psDocuments.Count > 0)

               {

                   for (int i = 0; i < psDocuments.Count; i++)

                   {

                       string _number = DateTime.Now.ToString("s").Replace("-", "").Replace(":", "").Replace("T", "");

                       Random randon = new Random();

                       _number += randon.Next().ToString();

                       string _fileName = _number + i.ToString() + "." + psDocuments[i].extension;

                       FileStream _streamWriter = new FileStream(filePath + _fileName, FileMode.Create);

                       _streamWriter.Write(psDocuments[i].binaryDataInBytes, 0, psDocuments[i].binaryDataInBytes.Length);

                       _streamWriter.Close();

                       _filesToMerge.Add(_fileName);

                   }

               }

           }

           catch(Exception ex)

           {

               throw ex;

           }

           finally

           {

           }

           return _filesToMerge;

       }

   }

}

using System;

using System.Collections.Generic;

using System.Linq;

using System.Web;

using System.Runtime.InteropServices;

using System.Runtime.CompilerServices;

namespace PsLiveDocX.WsHandler

{

   [ComImport(), Guid("00000016-0000-0000-C000-000000000046"),

   InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIUnknown)]

   public interface IMessageFilter

   {

       [PreserveSig]

       int HandleInComingCall(int dwCallType, IntPtr hTaskCaller, int dwTickCount, IntPtr lpInterfaceInfo);

       [PreserveSig]

       int RetryRejectedCall(IntPtr hTaskCallee, int dwTickCount, int dwRejectType);

       [PreserveSig]

       int MessagePending(IntPtr hTaskCallee, int dwTickCount, int dwPendingType);

   }

   public class MessageFilter : IMessageFilter

   {

       // Class containing the IMessageFilter

       // thread error-handling functions.

       // Start the filter.

       public static void Register()

       {

           IMessageFilter oldFilter = null;

           IMessageFilter newFilter = new MessageFilter();

           CoRegisterMessageFilter(newFilter, out oldFilter);

           System.Threading.Thread.Sleep(2000);

       }

       // Done with the filter, close it.

       public static void Revoke()

       {

           IMessageFilter oldFilter = null;

           CoRegisterMessageFilter(null, out oldFilter);

       }

       // IMessageFilter functions.

       // Handle incoming thread requests.

       int IMessageFilter.HandleInComingCall(int dwCallType, System.IntPtr hTaskCaller, int dwTickCount, System.IntPtr lpInterfaceInfo)

       {

           //Return the flag SERVERCALL_ISHANDLED.

           return 0;

       }

       // Thread call was rejected, so try again.

       int IMessageFilter.RetryRejectedCall(System.IntPtr hTaskCallee, int dwTickCount, int dwRejectType)

       {

           if (dwRejectType == 2)

           // flag = SERVERCALL_RETRYLATER.

           {

               // Retry the thread call immediately if return >=0 &

               // <100.

               return 99;

           }

           // Too busy; cancel call.

           return -1;

       }

       int IMessageFilter.MessagePending(System.IntPtr hTaskCallee, int dwTickCount, int dwPendingType)

       {

           //Return the flag PENDINGMSG_WAITDEFPROCESS.

           return 2;

       }

       // Implement the IMessageFilter interface.

       [DllImport("Ole32.dll")]

       private static extern int CoRegisterMessageFilter(IMessageFilter newFilter, out IMessageFilter oldFilter);

   }

}

using System;

using System.Collections.Generic;

using System.Linq;

using System.Web;

using System.Xml.Serialization;

using Microsoft.Office.Interop.Word;

namespace PsLiveDocX.WsHandler

{

   [Serializable]

   public class ResultStatus

   {

       public ResultStatus()

       {

           this.Messages = new List<string>();

           this.IsSuccessful = false;

       }

       public bool IsSuccessful;

       public List<String> Messages;

       public string TimeTakenInSecs;

   }

   public class Authentication

   {

       [XmlElement(IsNullable = true)]

       public string Username;

       [XmlElement(IsNullable = true)]

       public string Password;

       [XmlElement(IsNullable = false)]

       public string IsSecure;

   }

   public class AppendDocumentRequest

   {

       public AppendDocumentRequest()

       {

           AuthInfo = new Authentication();

           Documents = new List<PsDocument>();

           InsertPageBreaks = false;

       }

       public Authentication AuthInfo;

       public List<PsDocument> Documents;

       public Boolean InsertPageBreaks;

   }

   public class PsDocument

   {

       public string binaryData;

       public string extension;

       internal byte[] binaryDataInBytes;

   }

   public class AppendWordDocumentResult

   {

       public AppendWordDocumentResult()

       {

           ResultInfo = new ResultStatus();

       }

       public ResultStatus ResultInfo;

       public string OutputFileByteArray;

   }

}

cphilpot wrote re: Using a MessageFilter to avoid IDE threading and timing issues.
on 02-10-2011 9:54 AM

I've recently implemented this solution and it seems to work.  At first it did not work properly because the register was not being called on a STA thread.  Although on a quick glance that doesn't appear to be a problem with Dhananjay's code.

I've implemented a couple changes that I would recommend reviewing.  These are based on the use case of a user having a dialog open in the editor (e.g. filedialog is open).  

1.  Change the return value from 99 to 100 in IMessageFilter.RetryRejectedCall

       - This prevents 100% CPU usage

2.  Timeout after a period of time in IMessageFilter.RetryRejectedCall - e.g. if dwTickCount > 10000 //10 seconds then return -1

       - If after 10 seconds of retrying an operation it's probably best to inform the user with a suitable message (maybe suggest closing any open dialogs)

adamtarshis wrote re: Using a MessageFilter to avoid IDE threading and timing issues.
on 03-23-2011 2:23 AM

So this code works great in .net 3.5 targeted projects - but not in .net 4.  The Retry method is never called - it seems like the CoRegister method is not registering the filter.  Anyone run into this issue?

Copyright Summit Software Company, 2008. All rights reserved.