Creating timestamps for light stimulus in the video

[vgupta558]

Hello!

I am performing some optogenetic experiments where I am using arudino to turn on high power LEDs while an animal is in a harness and then recording the behavior. I would like to detect the frames when the light comes on in the video. I am wondering if there is a way that I can threshold the region of interest (where the illumination from the LEDs can be clearly detected) and create timestamps in a csv format that can be used with the tracking data I have from deeplabcut. I have never used Bonsai before so any help would be greatly appreciated. Happy to provide any additional information you may need for suggestions.

Thanks!
Vaidehi

[banchan86]

Hi @vgupta558 this tutorial exercise will give you what you need. Since you are aligning with the tracking data from deeplabcut, i suggest using an ElementIndex after GreaterThan. ElementIndex will yield both the frame number and a boolean which you can then save using a CsvWriter and integrate with your deeplabcut output.

[vgupta558]

Thank you! I will try this out and update!

[vgupta558]

I have one more question @banchan86 . Is there a way to combine the data from predict pose csv and the csv detecting LED state in bonsai itself? I can do it post processing but it would be really helpful if I can have the data combined during the same analysis run.

[banchan86]

I personally don't have experience with the deeplabcut library if you are using it for online pose estimation, but in theory it should work. You just need to Zip the items you want before writing to your CSV. Its important that you make sure the items being zipped have the same number of elements (meaning don't discard any frames or deeplabcut output). There might be a bit of buffering (memory usage) if the two sequences are not being produced at the same time and one has to wait for another, but it might be fine I think as long as the online pose estimation is not lagging too much.