The FRENDS4 architecture is meant to be distributed and cloud-ready. The Processes are executed by individual Agent services, which are grouped to Environments. There is also a centralized user interface for editing and monitoring the system, as well as a centralized message processing service. All these components may be hosted in the cloud or on-premise, depending on requirements.
The different parts of a FRENDS system communicate with each other using a messaging bus. Currently only Azure Service bus and Service bus for Windows Server are supported.
Because the message size in the Azure Service bus and Service bus for Windows Server is limited (~256 KB), the system will use a separate Large Message Store for passing larger messages. Usually the store uses Azure Blob Storage; for on-premise installations, you can also use file shares.
Currently, the following system queues are used:
- For communication from the UI to the Agents:
frends_configurationupdate_<agent environment name>_<agent name>: for sending configuration update messages, i.e. details of new Processes and versions to use
frends_routinetriggers_<agent environment name>: for triggering individual Process executions with "run once" messages. Common queue for all Agents
frends_routineinstance_<agent environment name>_<agent name>: for sending Process instance termination messages to a specific Agent that is running the Process instance
- For communication from the agents to the Message service
frends_heartbeatlog: for Agent heartbeats
frends_processlog: for Process execution details
- For communication from the Message service to the UI
frends_processnotifications: for sending notifications to the UI that some Process log data has changed, so it can update the view
An Agent is a process that executes the actual FRENDS Processes. It can be deployed as a Windows service, or in the cloud on a VM.
The Agent service also runs and hosts any Triggers used by the Processes, like HTTP Triggers or File watch triggers. This means that e.g. the HTTP Trigger URL should point to the agent host and configured port.
The Agent service will require an SQL instance to host the latest configuration and run-time data. For a simple single-node installation, SQL Server LocalDB is supported. For high availability, you can install the Agents in a farm, pointing to a single (clustered) SQL Server instance. in the cloud, you can use an (Azure) SQl database.
The Agent service also requires access to the Master package repository for downloading the needed Process and Library packages. When a Process is deployed to the Agent, The Agent will download the NuGet for the Process, as well as any referenced Task and library NuGets and expand them under its own process store. All the assemblies required for executing a Process are stored under a single directory, which will be used as the base directory for the AppDomain used for executing the Process.
Agents are always assigned to Environments, the logical configurations that use specific process versions and environment variables. All Agents in an environment share a single database and run the same Processes: if an Environment has multiple Agents, you cannot control which Agent will run a specific Process - for that, you need to define a separate Environment.
Process and other NuGet stores
When you create a Process, it is compiled to a .NET assembly and packaged as a NuGet package. These packages are then stored to the Master package repository, which contains all Process and Task NuGet packages, as well as any referenced library NuGet packages. The Master package repository uses Azure Blob Storage, but for on-premise installations, you can also use a file share.
In addition to the Master process store, also the Agents have their own Process store (located at
%AppData%\Local\HiQ Finland\processes for the Agent service user). There it stores the actual assemblies used for executing the processes.
The UI also has its own cache of all Task and library NuGets, stored under
~/App_Data. It needs these for compiling the Processes.
You create and edit new Processes, deploy them to Environments, monitor Process execution etc. with the centralized user interface. There is only one user interface for a FRENDS system. It can manage and monitor multiple Environments, e.g. Dev, Test, Staging and Production Environments, all on the same UI.
The user interface also updates the master configuration database (ConfigurationStore), which contains all the process details and old versions. As with the agent, for local installations, SQL Server is supported, and for cloud installations, the (Azure) SQL Database is to be used.
The UI will allow you to you to
- Create and edit processes, triggers and environment variables
- Deploy process versions between environments
- View the details of processes
- There will be extra support for viewing the status of Cobalt file transfers, including the latest successful and failed transfers
- View the list of Agents deployed in the network, and when their last heartbeat was received.
The Message processor is the background process that receives all log and heartbeat messages sent by the Agents from the Service bus. It processes and stores the data in the log database (LogStore), and can also send notifications to the UI.
The Message processor is also responsible for pruning the dead letter queues. The dead letter queues are special queues that are generated for each service bus queue and topic. If a message cannot be processed after retries, it is taken out from the original queue and pushed to the dead letter queue, with the error details. As the messages in the dead letter queue still consume the bus size quota, they need to be pruned periodically. By default, the Message processor checks all dead letter queues once per hour, and removes all but the latest 1000 dead-letter messages.
The Message processor also clears old history data from the process log database, by default after 60 days.
This Message processor is installed as a Windows service for on-premise installations. When deployed to Azure, it is executed as an Azure Web Job.
Below is a deployment diagram of an example system. The web site and Worker Roles / background are hosted in Azure, the Agent in the customer's server.
Each tenant uses its own Service bus namespace, with its own users and authentication, so the tenant-side components only have access to that tenant's messages.