Data flow bug in GH


(Olivier Stocker) #1

I’m trying to use genetics algorithms (GA), but i got a bug where each time the solver change the gene pool, it record the previous fitness.
though my research it seems it’s because there was 4 “ticks”, or “calculus shockwave” (i don’t know the real word) where the first still using the previous gene pool data, and only the second relies of the new genes. Then, the GA only reads the first one. At the end, i’m just facing wrong association of fitness and genes, resulting in a non-converting problem.

I tried to get back to the “split tick” source, which i blamed on “stream gates” from heteroptera plug-in. After deleting them from my gh, i still had 2 ticks. These last split seems to come right after the gene pool, but i wasn’t able to locate it neither understand what the hell is happening.

I used “data record” to register each tick. Hre is the result

(sorry for the mess, here is the order :

  1. gene pool
  2. split list (if more than 15 genes)
  3. data
  4. evaluate curve given a stable pack of curve (only 1 tick from it)
  5. list suppression of 1st and last element
  6. interpolate curve from the latice point
  7. pass it to other component to calcul the fitness (like lenght component)

My questions are :

  1. what can produce that double tick ?
  2. how to counter it, and have only 1 final fitness that correspond truly to the gene pool current settings ?
  3. why the output of the interpolate (then curve) componnent only present 1 tick
  4. is there more litterature on how data flow though components ?
    => i only found this

(qythium) #2

You might want to take a look at this blog post:

Basically after the initial expiration ‘shockwave’ the components are calculated in back-to-front order, triggering calculations from their expired upstream components.

This gets quite complicated if you work with components that store data between solutions or remotely trigger solution refreshes and changes in other components, as I believe most of these genetic-algorithm plugins behave.

However It’s probably not a good idea to rely on the invisible back-to-front ordering for the correct operation of your code, as that can easily change when copying and refactoring parts of your definition.

Have you tried using the Data Dam component to explicitly partition your definition into stages? This way you can be sure that whatever is on the left of the dam is entirely finished before proceeding to the right side.

(Olivier Stocker) #3

I just got my hand on it this morning ! it’s helped my understand how the triggers are planned and this order of component ! but thanks again :wink:

When GA are processing i can’t have my hand on it, so a data dam, which are manually activated component can’t be allowed.

I maybe just need a way to check what is creating looped shockwave.

(David Rutten) #4

But Data Dams will mess with Galapagos as well. It won’t know to wait for a dam.

(qythium) #5

That’s true :confused: I mostly run into these issues with the Kangaroo solver and having to do expensive refreshes on the canvas multiple times to get data to propagate fully - in that context Data Dams really help with understanding and controlling the flow.

@hobbes.stocker What are these mysterious looking components?

And I noticed some invisible wire inputs, it’s hard to tell what’s going on from a screenshot alone.

(Olivier Stocker) #6

It’s kind of strange : the “re-calculation” moment seems to not be stable over, this time it comes from an other parts of the GH file

The vba component only calculate once and contains :

If x <= 2 Then
A = 0
A = 1
End If

I tried to test this part in stand-alone, and everything is normal.

@qythium these are “heteroptera” component, i already use stream gates, but their “streaming” component family contains other element i tried, to solve my problem.
@DavidRutten have you an idea of what is happening ?


Can you replace the VB script for a ‘larger than’ component, just to see what happens. It should be capable of producing the same functionality and the problem starts right after. Just a test.

(Olivier Stocker) #8

@Roland_Schagen no changes. It seems to be deepest problem

(Olivier Stocker) #9

I finally found which component made all that mess :
the stream freeze/gate component from heteroptera is the one !
by replacing them with the “stream filter” component, I finaly have 1 expiration wave after each actualization of my gene pool, thus, only one fitness value per actualization. Now fitness value and gene pool are again linked together !

My coworker just told me he never had this kind of problem with this component. It’s maybe a update or something that create that.

Robert vier (octopus main dev) gave me really cool info about how the all thing work : so check also this post : Octopus : Uncomprehensive results