Assistance Required for Integrating Reinforcement Learning Model with Typhoon HIL

Question

I'm currently working on a project that aims to replace conventional LVRT mechanisms using reinforcement learning. I have successfully developed the reinforcement learning environment using Python to simulate LVRT. However, I am facing challenges in integrating this RL model with the simulation environment in Typhoon HIL. As far as I understand, the custom block in Typhoon requires code in C, whereas reinforcement learning models are typically implemented in Python. I am seeking your guidance on how I can integrate the machine learning/reinforcement learning model into the Typhoon HIL simulation environment, given this constraint. Your assistance would be greatly appreciated.

Answer 1

Do you want to run the RL constantly, or do you want to use it to tune certain control gains and leave them fixed? If the latter, you could run the RL in Python, and have the control gains as SCADA Inputs in the model. Use the Typhoon API to run the RL to define a set of gains, set the SCADA Inputs, capture relevant data, compute the metrics, and start another iteration.

If you are going this route, I would also suggest using TyphoonTest, so you can more easily write your tests and get a better report.

Answer 2

As you mentioned, you want to import the learning model based on Python language, you don't require to make changes in Schematic Editor model using C function block. There are three steps which can be followed for import to SCADA Panel:

1) The input which you want to import should be associated with SCADA Input component in the schematic model file which in turn impacts the system.
2)Once the model is complete, simply compile and load the model to HIL Scada. In the Scada Panel Initialization you can define the import file, path, directory and read file commands.
3)Then you need to link the defined initialization file with the Action Widgets available in the SCADA Panel file as per your requirement.

Answer 3

Greetings,

In addition to the great answers already posted here, I would like to add that the IEEE 35 node islanding with Artificial Neural Network model is posted on Typhoon Marketplace, which you can find on this link.

This model might be interesting for your case since the control algorithm is actually a Python script run from SCADA. Therefore, it might help you set up your own SCADA in a similar way.

Hope this helps,

Answer 4

Hi Bikas, welcome to the forum. Is your intention to use an actual HIL device in this project? Are RL model + LVRT comprising a controller model that you plan to load onto some DSP/hardware target?