Out of Process Workers in Dotnet

---
title: C# Out of Process Workers
author: Alan D. Jackson
date: May 29, 2013
source: https://alandjackson.wordpress.com/2013/05/29/c-out-of-process-workers/
---

Anytime you are writing a memory intensive task that will be executed over and
over, out of memory exceptions can become a concern (either because of memory
fragmentation or because it is in a host process such as IIS).

The pattern to run something in a new process is pretty easy and consists of a
few steps:

1. Serialize the worker information
2. Call an exe process and pass the serialized worker info
3. The exe process should deserialize and run the worker then serialize the results
4. The calling process deserializes the results and returns

## Utility Classes to Run an Executable

These classes are are required to get this working:

- [XmlSerializationUtil.cs](#xmlserializationutil.cs)
- [OutOfProcessRunner.cs](#outofprocessrunner.cs)

### XmlSerializationUtil.cs

```cs
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.Xml;
using System.Xml.Serialization;

namespace Util
{
    public class XmlSerializationUtil
    {
        public static string ToXmlString(object item)
        {
            var serializer = new XmlSerializer(item.GetType());
            using (var ms = new MemoryStream())
            using (var tw = new XmlTextWriter(ms, Encoding.UTF8))
            {
                tw.Formatting = Formatting.Indented;
                serializer.Serialize(tw, item);
                return UTF8Encoding.UTF8.GetString(ms.ToArray());
            }
        }

        public static T FromXmlString<T>(string xml)
        {
            xml = new Regex(@"^.*<\?xml.*?\?>\s*").Replace(xml, "");
            var serializer = new XmlSerializer(typeof(T));
            var bytes = UTF8Encoding.UTF8.GetBytes(xml);
            using (var ms = new MemoryStream(bytes))
            using (var tr = new XmlTextReader(ms))
            {
                return (T)serializer.Deserialize(tr);
            }
        }
    }
}
```

### OutOfProcessRunner.cs

```cs
using System;
using System.Diagnostics;
using System.IO;

namespace Util
{
    public interface IRunnable<T> { T Run(); }

    public class OutOfProcessRunner<T, U>
            where T : IRunnable<U>
    {
        public U RunOutOfProcess(string exePath, T runnable)
        {
            string stdin = XmlSerializationUtil.ToXmlString(runnable);

            var p = Process.Start(new ProcessStartInfo(exePath)
            {
                CreateNoWindow = true,
                UseShellExecute = false,
                RedirectStandardInput = true,
                RedirectStandardOutput = true,
                RedirectStandardError = true
            });
            p.StandardInput.Write(stdin);
            p.StandardInput.Close();
            var resultsStr = p.StandardOutput.ReadToEnd();
            p.WaitForExit();
            if (p.ExitCode == 0)
                return XmlSerializationUtil.FromXmlString<U>(resultsStr);
            else
                throw new ApplicationException("Error " + p.ExitCode + ": " + p.StandardError.ReadToEnd());
        }

        public string RunFromInput(string xml)
        {
            var item = XmlSerializationUtil.FromXmlString<T>(xml);
            var results = item.Run();
            var resultsStr = XmlSerializationUtil.ToXmlString(results);
            return resultsStr;
        }

        public int ExecuteRunnerProgram()
        {
            try
            {
                System.Console.Write(RunFromInput(ReadStdIn()));
                return 0;
            }
            catch (Exception e)
            {
                System.Console.Error.Write(e.ToString());
                return -1;
            }
        }

        static string ReadStdIn()
        {
            using (var s = System.Console.OpenStandardInput())
            using (var sr = new StreamReader(s))
                return sr.ReadToEnd();
        }
    }
}
```

## Modifying the worker process

The worker process needs to implement `IRunnable` and have a couple of utilities
to get a runner:

```cs
using Util;

namespace AlanDJackson.Work
{
    public class MyWorkerResults
    {
        public string ResultsMessage { get; set; }
    }

    // This extension lets you call "new MyWorker().RunOutOfProcess()"
    public static class MyWorkerExtensions
    {
        public static MyWorkerResults RunOutOfProcess(this MyWorker worker)
        {
            return MyWorker.Runner.RunOutOfProcess("MyWorker.exe", worker);
        }
    }

    public class MyWorker: IRunnable<MyWorkerResults>
    {
        public static OutOfProcessRunner<MyWorker, MyWorkerResults> Runner =
            new OutOfProcessRunner<MyWorker, MyWorkerResults>();

        public MyWorkerResults Run()
        {
            return new MyWorkerResults() { ResultsMessage = "OK" };
        }
    }
}
```

## Creating the worker exe

Create a new console application and add a reference to the assembly with your
worker process. This single line with use the runner utility to run the worker
from stdin and write the output to stdout (or stderr in the case of an error).

```cs
using Work;

namespace AlanDJackson.Work.Console
{
    class Program
    {
        static int Main(string[] args)
        {
            return MyWorker.Runner.ExecuteRunnerProgram();
        }
    }
}
```

## Also see...

- [Long Running Background Tasks in Asp.Net MVC3](https://alandjackson.wordpress.com/2012/04/11/long-running-background-tasks-in-asp-net-mvc3/)
    - [Example Github Project](https://github.com/alandjackson/AsyncTask)