Process logs in Workflow Engine

Overview

There are several types of logs and logging functionalities in Workflow Engine as follows:

• the process instance log where messages about changes in the state of the process instances are written. It can be enabled/disabled booth at the level of the process scheme and at the level of a specific process instance.
• the WorkflowRuntime log where messages from WorkflowRuntime (i.e start/stop, timers, process restore etc.) are written.
• the WorkflowRuntime events such as OnWorkflowError, ProcessActivityChanged and ProcessStatusChanged that allow to customize the logs completely.
• process history log in WorkflowProcessTransitionHistory table.

Process instance log

Enabling process instance logs

Enabling the logs can be done in the scheme editing interface in the Designer for all processes and in a particular scheme. Logging function is disabled by default, but you can set it by the Logs or Process Info button - a window will be opened where you can set this parameter. After changing the parameter, the scheme must be saved.

View of the process instance log records

In the process instance view interface, it is possible to enable (or disable) the logs for a specific process instance.

When the process instance is run, the logging settings become available also by clicking on Logs button in the Designer toolbar. Then, the current process logging records can be browsed in the new window and the following options will be displayed:

1. The logging options can be enabled or disabled for the further processes.
2. Update, pulling up new data, if there is any.
3. Allowing auto-refresh.
4. Setting auto-refresh period.
5. Timestamp - which contains information about the time when the event occurred;
6. Message - message containing basic information about the event;
7. Exception - error data when it is occurred.
8. The (i) visual indicator in the row - it displays the detailed logging information when positioning the mouse over it.

The process logs are registered by IProcessLogProvider that writes the log records to the memory in the default implementation, so it might be used for debugging, but to store logs, you should write your own implementation of IProcessLogProvider.

Process logs implementation

The IProcessLogProvider is implemented to register process logs, an example is provided in MemoryProcessLogger class which is used by default.

IProcessLogProvider

The IProcessLogProvider interface to interact with logging can be added, and it is accessed through WorkflowRuntime.

public interface IProcessLogProvider
{
    void Write(ProcessLogEntry entry);
        
    /// <summary> Get all records for specific process. </summary>
    /// <param name="processId">Id of <see cref="Model.ProcessInstance"/></param>
    Task<IEnumerable<ProcessLogEntry>> ReadAllAsync(Guid processId);
        
    /// <summary> Get last records starting the specified time </summary>
    /// <param name="processId">Id of <see cref="Model.ProcessInstance"/></param>
    /// <param name="time"></param>
    /// <returns></returns>
    Task<IEnumerable<ProcessLogEntry>> ReadLastAsync(Guid processId, DateTime time);
    /// <summary> Get last records in certain amount </summary>
    /// <param name="processId">Id of <see cref="Model.ProcessInstance"/></param>
    /// <param name="count">Count of records</param>
    /// <returns></returns>
    Task<IEnumerable<ProcessLogEntry>> ReadLastAsync(Guid processId, int count);

    /// <summary> Get earle records starting the specified time in certain amount </summary>
    /// <param name="processId">Id of <see cref="Model.ProcessInstance"/></param>
    /// <param name="time"></param>
    /// <param name="count">Count of records</param>
    /// <returns></returns>
    Task<IEnumerable<ProcessLogEntry>> ReadEarlyAsync(Guid processId, DateTime time, int count);        
}

The ProcessLogger is used to register process logs. It is possible to connect a custom process logger to WorkflowRuntime through the WithProcessLogger method:

workflowRuntume.WithProcessLogger(new MyCustomProcessLogger (...)); 

Besides, an implementation example of IProcessLogProvider is given in the class ProcessMemoryLog.

ProcessMemoryLog

The implementation of IProcessLogProvider interface is added by default in ProcessMemoryLog, it stores logs in memory while the server is running.

The maximum number of logged processes and the maximum number of stored records are defined in the constructor during initialization. 100 processes and 100 records are set by default.

In contrast to the number of logged processes, records are stored in a queue, and the old records are deleted when new ones appear.

The IProcessLogProvider implementation in this case:

public class MemoryProcessLogger : IProcessLogProvider
{
    /// <summary> Maximum number of log records in one process. </summary>
    public int LogSize { get; }
    /// <summary> Maximum number of logged processes. </summary>
    public int Capacity { get; }

    public ConcurrentDictionary<Guid, FixedSizedQueue<ProcessLogEntry>> ProcessLogs { get; } =
        new ConcurrentDictionary<Guid, FixedSizedQueue<ProcessLogEntry>>();

    /// <summary>
    /// Keep a fixed number of new logs by deleting old ones for fixed number of process in memory.  
    /// </summary>
    /// <param name="logSize">Maximum number of log records in one process.</param>
    /// <param name="capacity">Maximum number of logged processes.</param>
    public MemoryProcessLogger(int logSize, int capacity)
    {
        LogSize = logSize;
        Capacity = capacity;
    }

    public void Write(ProcessLogEntry entry)
    {
        if (ProcessLogs.Count >= Capacity) { return; }
        InitQueue();
        ProcessLogs[entry.ProcessId].Enqueue(entry);

        void InitQueue()
        {
            if (!ProcessLogs.ContainsKey(entry.ProcessId))
            {
                    ProcessLogs[entry.ProcessId] = new FixedSizedQueue<ProcessLogEntry>(LogSize);
            }
        }
    }

    public async Task<IEnumerable<ProcessLogEntry>> ReadAllAsync(Guid processId)
    {
        return ProcessLogs.ContainsKey(processId)
            ? ProcessLogs[processId].Where(entry => entry.ProcessId == processId)
            : new List<ProcessLogEntry>();
    }

    public async Task<IEnumerable<ProcessLogEntry>> ReadLastAsync(Guid processId, DateTime time)
    {
        return ProcessLogs.ContainsKey(processId)
            ? ProcessLogs[processId].Where(entry => entry.ProcessId == processId && entry.CreatedOn > time)
            : new List<ProcessLogEntry>();
    }

    public async Task<IEnumerable<ProcessLogEntry>> ReadLastAsync(Guid processId, int count)
    {
        return ProcessLogs.ContainsKey(processId)
            ? ProcessLogs[processId]
                .Where(entry => entry.ProcessId == processId)
                .OrderByDescending(entry => entry.CreatedOn)
                .Take(count)
            : new List<ProcessLogEntry>();
    }

    public async Task<IEnumerable<ProcessLogEntry>> ReadEarlyAsync(Guid processId, DateTime time, int count)
    {
        return ProcessLogs.ContainsKey(processId)
            ? ProcessLogs[processId]
                .Where(entry => entry.ProcessId == processId && entry.CreatedOn < time)
                .OrderByDescending(entry => entry.CreatedOn)
                .Take(count)
            : new List<ProcessLogEntry>();
    }
}

ProcessLogEntry

The class ProcessLogEntry is used as presentation data model for the Designer:

public class ProcessLogEntry
{
    public Guid ProcessId { get; set; }
    public DateTime CreatedOn { get; set; } = DateTime.Now;
    public DateTime Timestamp { get; set; }
    public string Message { get; set; }
    public JObject Properties { get; set; }
    public Exception Exception { get; set; }
}

Indeed, the process logs are sent and received through this class.

The registered message depends on the events that occurred, which are identified in the ActivityExecutionEventType enumeration. The execution time of the event is added at the end of the Message.

The listing ActivityExecutionEventType.

public enum ActivityExecutionEventType
{
    StartExecution,
    EndExecution,
    ResumeExecution,
        
    InitializeProcess,
    InitializeSubProcess,
        
    SelectActivity,
        
    ExecuteCustomActivity,
    CallAction,
    CallCondition,
    CallActionCondition,
    CallActionFromProvider,
    CallConditionFromProvider,
        
    FoundAlwaysCondition,
    FoundCondition,
    FoundOtherwiseCondition,

    ActionNotImplemented,
    ActionConditionNotImplemented,
    ActivityExecutionError,

    InverseCondition,

    ExecuteTransition,

    IdleTimeout,
    ExecuteAttempt,
    HandlingTimeout,
    HandlingException,
        
    TriggerTransition,
    SetActivityWithExecution,
        
    SetState,
    SetActivity
}

All logs are generated when the following classes are executed: ActivityExecutor and ProcessExecutor.

take into account

More details related to these structures are provided in the API Reference section.

ProcessLogEntry and ActivityExecutionEventType.

WorkflowRuntime log

Workflow Engine provides not only process logs but also events logs that take place in WorkflowRuntime. If you want to customize this log, the Ilogger interface must be implemented.

ILogger

The ILogger interface can register information related to errors in the WorkflowRuntime, the timers execution and so on.

It is used as indicated below:

public interface ILogger : IDisposable
{
    void Debug(string messageTemplate);
    void Debug(string messageTemplate, params object[] propertyValues);
    void Debug(Exception exception, string messageTemplate);
    void Debug(Exception exception, string messageTemplate, params object[] propertyValues);

    void Error(string messageTemplate);
    void Error(string messageTemplate, params object[] propertyValues);
    void Error(Exception exception, string messageTemplate);
    void Error(Exception exception, string messageTemplate, params object[] propertyValues);

    void Info(string messageTemplate);
    void Info(string messageTemplate, params object[] propertyValues);
    void Info(Exception exception, string messageTemplate);
    void Info(Exception exception, string messageTemplate, params object[] propertyValues);
}

Moreover, ILogger might be used with WorkflowRuntime to log events from the engine. An implementation example is provided in the class ConsoleLogger.

public class ConsoleLogger : ILogger
{
    public bool LogInfo { get; set; }
    public bool LogDebug { get; set; }
    public bool LogError { get; set; }
        
    public void Dispose()
    {
    }

    public static ConsoleLogger LogAll => new ConsoleLogger() {LogInfo = true, LogDebug = true, LogError = true};
        
    public static ConsoleLogger LogErrors => new ConsoleLogger() {LogInfo = false, LogDebug = false, LogError = true};
        
    public void Debug(string messageTemplate, params object[] propertyValues)
    {
        if (!LogDebug)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("DEBUG----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }                    

    public void Debug(string messageTemplate)
    {
        if (!LogDebug)
        {
            return;
        }

        var sb = new StringBuilder();
        sb.AppendLine("DEBUG----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Debug(Exception exception, string messageTemplate)
    {
        if (!LogDebug)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("DEBUG----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine(exception.ToString());
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Debug(Exception exception, string messageTemplate, params object[] propertyValues)
    {
        if (!LogDebug)
        {
            return;
        }

        var sb = new StringBuilder();
        sb.AppendLine("DEBUG----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine(exception.ToString());
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Error(Exception exception, string messageTemplate, params object[] propertyValues)
    {
        if (!LogError)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("ERROR----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine(exception.ToString());
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Error(string messageTemplate)
    {
        if (!LogError)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("ERROR----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Error(Exception exception, string messageTemplate)
    {
        if (!LogError)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("ERROR----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine(exception.ToString());
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Error(string messageTemplate, params object[] propertyValues)
    {
        if (!LogError)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("ERROR----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Info(string messageTemplate, params object[] propertyValues)
    {
        if (!LogInfo)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("INFO----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Info(Exception exception, string messageTemplate, params object[] propertyValues)
    {
        if (!LogInfo)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("INFO----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine(exception.ToString());
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Info(Exception exception, string messageTemplate)
    {
        if (!LogInfo)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("INFO----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine(exception.ToString());
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Info(string messageTemplate)
    {
        if (!LogInfo)
        {
            return;
        }
            
        var sb = new StringBuilder();
        sb.AppendLine("INFO----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {messageTemplate}");
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Info(string eventSource, string messageTemplate, params object[] propertyValues)
    {
        if (!LogInfo)
        {
            return;
        }

        var sb = new StringBuilder();
        sb.AppendLine("INFO----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {eventSource} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Error(string eventSource, Exception exception, string messageTemplate, params object[] propertyValues)
    {
        if (!LogError)
        {
            return;
        }

        var sb = new StringBuilder();
        sb.AppendLine("ERROR----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {eventSource} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine(exception.ToString());
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Error(string eventSource, string messageTemplate, params object[] propertyValues)
    {
        if (!LogError)
        {
            return;
        }

        var sb = new StringBuilder();
        sb.AppendLine("ERROR----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {eventSource} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }

    public void Debug(string eventSource, string messageTemplate, params object[] propertyValues)
    {
        if (!LogDebug)
        {
            return;
        }

        var sb = new StringBuilder();
        sb.AppendLine("DEBUG----------------");
        sb.AppendLine($"{DateTime.Now:dd.MM.yyyy HH:mm:ss.fff} {eventSource} {messageTemplate}");
        sb.AppendLine(JsonConvert.SerializeObject(propertyValues));
        sb.AppendLine("--------------------");
        Console.WriteLine(sb);
    }
}

public static class ConsoleLoggerExtensions
{
     public static WorkflowRuntime WithConsoleAllLogger(this WorkflowRuntime runtime)
     {
         runtime.Logger = ConsoleLogger.LogAll;
         SubscribeErrorHandler(runtime);
         return runtime;
     }

     private static void SubscribeErrorHandler(WorkflowRuntime runtime)
     {
         void RuntimeOnOnWorkflowError(object sender, WorkflowErrorEventArgs args)
         {
             runtime.LogErrorIfLoggerExists(args.Exception.Message,
                 new Dictionary<string, string>()
                 {
                     {"RuntimeId", runtime.Id},
                     {"ProcessId", args.ProcessInstance.ProcessId.ToString()},
                     {"RootProcessId", args.ProcessInstance.RootProcessId.ToString()},
                     {"IsSubprocess", args.ProcessInstance.IsSubprocess.ToLowerCaseString()},
                     {"ExecutedActivity", args.ProcessInstance.ExecutedActivity?.Name},
                     {"ExecutedTransition", args.ProcessInstance.ExecutedTransition?.Name},
                     {"Exception", args.Exception.ToString()}
                 });
         }

         runtime.OnWorkflowError += RuntimeOnOnWorkflowError;
     }

     public static WorkflowRuntime WithErrorLogger(this WorkflowRuntime runtime)
     {
         runtime.Logger = ConsoleLogger.LogErrors;
         SubscribeErrorHandler(runtime);
         return runtime;
     }
}

After implementing the logger, connect it to WorkflowRuntime as follows:

workflowRuntume.WithConsoleAllLogger();

Custom logging using events

OnWorkflowError

When calling on events from Workflow Engine, an application can write its own logs.

The OnWorkflowError event can be used to catch all errors as is indicated below:

runtime.OnWorkflowError += (sender, args) =>
{
    Exception exception = args.Exception;
    ProcessInstance processInstance = args.ProcessInstance;
    TransitionDefinition executedTransition = args.ExecutedTransition;
};

The event handler enables you to change a process state or send an error notification.

ProcessActivityChanged

The ProcessActivityChanged event (or their asynchronous versions) can be triggered to log every change in the activity of a process.

runtime.ProcessActivityChanged += (sender, args) => { };

ProcessStatusChanged

The ProcessStatusChanged event is used when changes related to the process status take place.

runtime.ProcessStatusChanged += (sender, args) => { };

Transition history log

Furthermore, the process transition history logs, can be found in WorkflowProcessTransitionHistory table in MS SQL database.

info

More information related to DB Entities can be read here.