Grasshopper, Reinforcement learning code

Dear all,
I want to connect my trained RL code with the grasshopper. I need to do optimization, but with my code. How can I connect my code with my grasshopper model?

Thanks in advance

Hello,

please give us more concrete information. What language/technology have you implemented that in? There are various ways to inject external code or to make use of Interprocess Communication. But in any case, it requires some effort and knowledge, so make sure it’s worth to do.

Dear all,
I have run honeybee simulation and obtain some initial results, I have adjusted the percentage of the adaptive facade from 10 percent to 90 percent.
My question is, I want to use RL techniques as an optimization, and then I need to address that this technique is a sufficient technique to control adaptive facades and optimize daylight.
But how do I know what kind of techniques have been used by energy plus to give me PMV or energy consumption?

Generally, I want to know if i use an RL controller, how can i prove and compare with other control techniques?I don’t know how a comparison would make sense

Hi -

That sounds like a question for the LadybugTools or Unmethours forum.
-wim

I don’t know and my question is how can I prove that RL controller is a sufficient way as a technique to control an adaptive facade?

This is really not easy to tell. In an industrial context, there are different levels of testing, called ‘XIL’ or ‘In-the-Loop’ testing. You have ‘Model in the Loop’, where you test a software component against a physical simulation. ‘Software in the Loop’, where you test your controller software on a virtual and non-virtual controller device, and you have the ‘Hardware in the Loop’ where you test if a component is actually stimulating a real device the way you want that. And finally you can test the full system. This sort of work is the daily business for many engineers within a company, so I don’t know if you will ever get an answer in a thread like this. :slight_smile: What you want is probably MIL testing, and that’s probably about how can you do integration, component and unit tests with GH and Ladybug. And here you enter uncharted terrain… In my understanding the whole GH environment is very poor in this regard.

Dear Tom,
Thanks for your explanations.
What I did is connecting my DRL model and grasshopper and obtained some results.
But I don’t know how to prove that , The DRL controller can help efficiency. Because all of my data comes from simulation and I do not have actual data.

Now I don’t know how can I represent my results. And how can I show it reduces the energy consumption.
I obtained the energy consumption from Energy plus but I don’t know how can I compare, because I don’t know what is the control techniques.

You can take several generic scenarios of different and common facades, and take the same metrics with the different optimisation techniques (calculation time, accuracy, different types of error, etc.) and put them in a table where the rows are the techniques and the columns are the metrics. You can use the standard/most popular technique as the reference. And you need a process for measuring these metrics that is independent of the technique. I don’t know anything about this kind of simulations so I can’t really help.

I don’t see how an algorithm like Q-learning is going to get you better results than using more straight to the point optimisation techniques. What RL has with respect to other techniques is that it allows you to solve other types of tasks, but as an optimisation technique it does not stand out. RL stands out when there are agents that you want to learn the right behaviour through trial and error, by reward. But if you find something interesting please share it.

I have seen people using supervised learning models learn to simulate and after training, although the accuracy drops a bit, the computation time is dramatically reduced. I don’t remember the source right now but it has been shared on this forum.

I still don’t fully understand the problem you have. A simulation is almost never accurate, especially if you don’t know how this simulation works. But a good simulation likely narrows it to a real world scenario. You can basically only prove it by doing a real test or by comparing it to previous simulations which were correct. But I also don’t understand the need for RL model? Shouldn’t the process be deterministic if you use sensors? Why do you need artificial intelligence for, if you can measure the actual daylight? But I also have no clue about this whole topic…

Dear Dani,
Thanks for your useful explanations,
If I am not mistaken, you propose, to do a simulation for example with Galapagos (Genetic Algorithm ) compare with DRL controller?

What if , I just show we can use these techniques as a new technique, And represent the results that we can obtain from my objective with the current technique.

I agree with you, this technique may not be as better as other techniques.

Dear Tom,
Thanks for your kind explanations.
I understand what are you saying.

GA’s are good at finding the best parameter set out of many different parameter sets by mutation and combination. It is not the kind of task of a light simulation.

The better technique should be regression or some multivariate optimisation or support vector machines or something like that.

What you would do with RL would be to optimise a model so that this model then optimises the simulation task, instead of directly optimise what you need. Another problem you will have is that with RL you will not be able to generalise, you will have to retrain your model for each facade (unless you have thousands of facades to train it with, which I don’t think is the case). If you compare RL results with an optimisation technique on one or two facades, you may not be able to demonstrate anything more than the results on those specific facades and not in general, because for this you need a model that learns to simulate in a supervised way.

I think he is not trying to optimize the CAD model, but the (micro-)controller to a dynamic/adaptive facade. Then RL makes sense, but only if you are lacking sensors to immediately react to a given situation. And here is the problem, how do you know that the trained model actually works correctly, if you have nothing to measure. At least this is my understanding of what has been written here.

Ah I see, so maybe I was wrong before. Thanks for the clarification although it is still unclear to me. If for example the objective is to block sunlight at a given percentage under any environment, the amount of light can be represented computationally to measure the error, as an area maybe, isn’t it? I don’t know, without detailing the problem with inputs and outputs it’s all just a lot of guesswork.

Thanks for the explanation

To explain more i want to control adaptive façade by RL to optimize daylight.
But is there any way to compare the benefit of this control technique?

Do you think it would be sufficient if with same objective i run simulation with octopus or Galapagos and then compare with RL controller?
What control techniques Galapagos and octopus using to control variables, I mean like windows, façade,…?

If i just represent this controller as a one techniques, how can i represent my results?

I like to keep things simple, so thats what I would do in your case: I would completely forget about Machine Learning and just buy some sensors.I think you need sensors to measure Photon Flux density (PFD/PPFD) if you care about Plants, the total illuminance with a plain lux sensor, and you could measure room temperature as well. And as soon as your sensor array measures low values, you are opening your facade to enter more light, stepwise, until you have the values you like. There is no magic to it. Isn’t it like you sitting at a window and if it’s too bright you just pull down the jalousie. I really see no reason to actually “improve” this by using ML. Maybe if some values are contradicting, like you want to keep it cool, but also bright enough to not becoming depressed. But then you just need to weight what’s more important. In the winter light is more important, in the summer it’s having cool room temperature. At least in temperate climate zones.

1 Like

Dear Tom,
I hope you are doing well. I have a question: I write my q_learning algorithm in Python 3, I have simulation results in Grasshopper, and I have 64 actions (3 sliders with 4 numbers), and I have simulation each hour as (state). My problem is how to update my state and actions. I have a Gh Python remote.