Will it be possible to use separate (identical) components to be running in different threads?
There’s a point in separating “orchestration” (= single thread) and the actual processing units (each component = one thread) because this keeps components simple®, and the layout on the canvas would convey the design. The number of threads in use to be considered being part of the design.
ParallelGroup would analyze the number of Jobs and dispatch only available number of threads to each Job component (if there are 12 Job components in the group, but only 6 threads available, then only 6-1 job components would run, due to reserving one thread for the main “orchestration” of the entire map-reduce network on canvas).
This would make GH definitions crystal clear and make it simpler to avoid data races and… well, this is actually the simplest and most generic way to handle concurrency, and it would work for any solutions. Single threaded data streams (part of orchestration) makes the concept dead simple to implement technically, and dead simple for users to configure and … well.
FBP (Flow Based Programming) has proven to work very well and demystifies parallel computing even making it available to average people. You can’t explain mutexes and data races to grandma while anyone can understand a layout of machines on the factory floor (resembling the processing GH Job Components). Grandma doesn’t have the problem of “dataraces” between her and her work mates’ two sewing machines when picking fabric from two different material pallets.
- The Canvas and all ParallelGroups = “Main thread”
- Components on ParallelGroups = 1 thread each, as many as available.
Additions to the GH Concept::
AbstractJobComponent (subclassing all other components…).
This reminds me about the concept of PlaneGroups (coordinating components to respect the same plane by placing them on the same group (
How about that?