This is part 3 of the Maestro for Drupal 8 blog series, defining and documenting the various aspects of the Maestro workflow engine. Please see Part 1 for information on Maestro's Templates and Tasks, and Part 2 for the Maestro's workflow engine internals. This post will help workflow administrators understand why Maestro for Drupal 8's validation engine warns about the potential for loopback conditions known as "Regeneration".
Loopbacks and Regeneration
Since 2003, Maestro had the concept of "Regeneration". Regeneration for Maestro on Drupal 7 was a rather nebulous concept if you didn't understand the complexities that exist within the original engine. Most administrators would simply check the regeneration option to "on" and the option to "recreate all in production tasks" and never worry about what it actually meant. Simply put, as a human reading a workflow diagram, we can understand when a logic condition causes a loop-back over already executed tasks. However, for a machine to detect when we're looping-back over already executed tasks produces a set of logical issues that are not easily overcome.
Unfortunately for the machine logic in the Drupal 7 Maestro engine, a loop-back over already executed tasks signaled that something is terribly wrong! It is a safety mechanism that prevented logic traps when executing a single task multiple times. The solution was to have an administrative option in Drupal 7 (and prior non-Drupal versions) to tell the engine that indeed we do want to re-create the task in the queue and that we would like it if the workflow could re-spawn itself with a wholly new process ID and copy over all in-production tasks and data. Regeneration was an evil necessity until recently.
We embarked on a project to simplify the engine execution of templates by trying to remove the notion of regeneration on logical loopbacks. The obvious thing we could have done was to hide the regeneration option and just have it turned on for each task. Problem solved! Well, sort of. Regeneration, while crafty in its approach, produces multiple processes linked via a linked list from the parent to its children. The cascading of processes caused confusion to new Maestro admins and is a concept that is hard to grasp except for those of us who have lived with the logic of regeneration for nearly 15 years.
The resolution to the regeneration problem is to re-code the engine such that it recognizes a logical loopback condition and tries to understand whether it should or should not create tasks that have already been created in the queue. The default scenario of "always regenerate" is now the default in the Maestro D8 engine, but it has some caveats explained further here.
Consider the workflow of the example Form Approval Flow provided with Maestro:
As shown, the content type task is the first task in the task chain that has the capacity to be executed multiple times during the workflow's lifetime. It is the first "real" task of the workflow (Start task is the first task, but is just a placeholder) and it can be looped over and over again until the form is approved. Maestro for D8 will now re-create the content type task as many times as required all without ever changing the process ID! This not only has simplified the administration of the engine, but also removes a point of possible failure if a regeneration copy of the process failed. While the tasks following the Content Type task are re-executed during a loopback, it is critical that each task doesn't flag the engine as being "regenerated", thus causing the engine to constantly flag and update the existing open process' tasks as regenerated. This too is handled by the new Maestro engine in Drupal 8.
Because this task has multiple pointers pointing to it from the Start task and from the Set Process Variable task, and the fact that it is not one of the two special task types of AND or OR (more on this later), a loopback condition is signaled in the engine. The engine automatically handles the loopback condition, preserving the process ID and all in-production and executed tasks in the queue.
When you validate the example Form Approval flow, you should see the warning message shown above. The message reads, "This task, with two pointers to it, will cause a regeneration to happen". This message is either confusing to you or is what you expect to happen anyway! What the validation engine is telling you is that because this is not an AND or OR task, the engine will recreate the already created once task for you in the queue. The validation engine is showing this as a warning/informational message only and will allow the template to be put into production.
Special Tasks - The AND and OR
The AND and OR tasks are the ONLY two tasks that the engine will allow two or more pointers to and will NOT signal a loopback condition. It is critical as a workflow designer to understand that if you wish to do a loopback to previously executed tasks, pointing to an AND or OR task will NOT produce a loopback scenario. Looping back to an AND task is logically incorrect to begin with as the AND task will never complete in the first place if one of its input pointers is coming from an area of the template that falls logically after the AND. Looping back to an OR task, while not logically incorrect, simply does not reproduce the OR. A loopback to an OR may simply just stall the workflow at that point as the OR has already been created in the queue and does not produce a loopback regeneration condition.
In short, excluding the start, end, OR and AND tasks, all other tasks produce a logical loopback condition and will signal the engine to re-create already created tasks in the queue for execution. The workflow administrator's job of figuring out where logical loopbacks may occur or which options to enable on the workflow are now a thing of the past. Maestro for Drupal 8 now takes care of the logic and provides the workflow administrator with validation messages, predicting where loopback conditions may occur.
