From 491a40628005af03fc8918e147eee9f3a53109ae Mon Sep 17 00:00:00 2001 From: cjsha Date: Fri, 1 Nov 2024 14:54:16 -0400 Subject: [PATCH 01/19] Add initial content for spike detection tut --- articles/tutorials/spikes.md | 131 ++++++++++++ articles/tutorials/toc.yml | 4 +- docfx.json | 3 +- .../tutorials/spikes/configuration.bonsai | 125 ++++++++++++ workflows/tutorials/spikes/ephys-data.bonsai | 26 +++ .../tutorials/spikes/spike-detection.bonsai | 75 +++++++ workflows/tutorials/spikes/spikes.bonsai | 188 ++++++++++++++++++ 7 files changed, 550 insertions(+), 2 deletions(-) create mode 100644 articles/tutorials/spikes.md create mode 100644 workflows/tutorials/spikes/configuration.bonsai create mode 100644 workflows/tutorials/spikes/ephys-data.bonsai create mode 100644 workflows/tutorials/spikes/spike-detection.bonsai create mode 100644 workflows/tutorials/spikes/spikes.bonsai diff --git a/articles/tutorials/spikes.md b/articles/tutorials/spikes.md new file mode 100644 index 00000000..ce29d51e --- /dev/null +++ b/articles/tutorials/spikes.md @@ -0,0 +1,131 @@ +--- +uid: spikes +title: Spike Detection +--- + + + +This tutorial is dedicated to teaching how to use ONIX hardware and the OpenEphys.Onix1 Bonsai package to detect spikes +with a simple fixed threshold as well as visualize and save raw/spike data. The following workflow is an example of +what you can expect after finishing this tutorial by the end of the tutorial. + +::: workflow +![/workflows/tutorials/spikes.bonsai workflow](../../workflows/tutorials/spikes/spikes.bonsai) +::: + +> [!NOTE] +> Although this tutorial uses the headstage64 to exemplify the spike detection process, it's pretty similar for +> other hardware. This tutorial in addition to the [hardware guide](xref:hardware) for your particular +> ONIX headstage should be sufficient for you to be able to detect spikes with whatever you have your hands on. + +1. [Get started](xref:getting-started) in Bonsai. In particular, + [download the necessary Bonsai packages](xref:install-configure-bonsai#install-packages-in-bonsai). + If you've already downloaded them, + [check for updates](xref:install-configure-bonsai#update-packages-in-bonsai). This tutorial assumes you're using the + latest software. + +1. Create the top-level configuration graph + + ::: workflow + ![/workflows/tutorials/spikes/configuration.bonsai workflow](../../workflows/tutorials/spikes/configuration.bonsai) + ::: + + In this workflow, all devices on the breakout board and all devices on the headstage64 except the Rhd2164 are + configured to be disabled at run-time. For the sole purpose of teaching how to detect spikes, I only need the data + from Rhd2164 Intan amplifier and don't have to bother with all that other stuff. + + If you are collecting electrophysiology data from hardware that is not the headstage64, replace the + node with a [configuration operator](xref:configure) that corresponds to + your hardware, and confirm the device that streams electrophysiology data on that headstage is enabled. + +1. Place the relevant operator to stream electrophysiology data from your hardware and select the + relevant output members + + ::: workflow + ![/workflows/tutorials/spikes/ephys-data.bonsai workflow](../../workflows/tutorials/spikes/ephys-data.bonsai) + ::: + + Because I am using headstage64 in this example and the headstage64 streams electrophysiology through Rhd2164 Intan + amplifier, I place the node onto the workflow. If you are using different + hardware, the table at the bottom of this tutorial[^1] provides a reference for which ephys you need + to place onto your workflow for your particular hardware and links to relevant documentation. Select the relevant + members from the frames that are produced by the chosen ephys data operators. In this case, those members are + "AmplifierData" and "Clock". This is most easily performed by clicking one of the members that appears after + hovering over the output option in the context menu that appears after right-clicking the `Rhd2164` node. + + + + Let's visualize the raw data. Start the workflow by pressing F5 or clicking the green triangle play + button at the top of the workflow editor. Double click the selected member that is streaming ephys data. In my case, + that's the `MemberSelector` selecting "AmplifierData". + + + + + +1. Create the data processing graph for spike detection + + ::: workflow + ![/workflows/tutorials/spikes/spike-detection.bonsai workflow](../../workflows/tutorials/spikes/spike-detection.bonsai) + ::: + + First, select from which channels you want to detect spikes by connecting a `SelectChannels` node to your + electrophysiology data stream and editing its "Channels" property. + + Center the dynamic range of the signal around zero and scale the ADC signal to μV. Although the following + calculation can be done in one `ConvertScale` operator, the calculations are more straightforward using two of them + connected in series because the `ConvertScale` operator applies the "Shift" offset after applying the "Scale" scalar + instead of the reverse order of operation. Connect the first `ConvertScale` operator to the `SelectChannels` operator. + - Center the dynamic range of the signal around zero by subtracting 2^bit depth - 1^ from the signal. This is done + in your Bonsai workflow by editing the "Shift" property of the first `ConvertScale` operator. If you are unsure + about the bit depth of the electrophysiology data from your particular hardware, refer to the table at the + bottom of this tutorial.[^1] The headstage64's Rhd2164 device has a bit depth of 16, so I set "Shift" to 32768 to + subtract 32768 from our data. I also set the "Depth" property to S16 so that the negative data is properly + represented. S16 refers to a signed 16-bit data type + - Scale the ADC signal to μV by multiplying all values in the ADC signal by a scalar that is determined by the + gain of the amplifier and resolution the ADC contained in the amplifier device This is done in your Bonsai + workflow by editing the "Scale" property of the second `ConvertScale` operator. If you are unsure about the step + size of the electrophysiology data from your particular hardware, refer to the table at the bottom of this + tutorial.[^1] The Rhd2164 amplifier device has a step size of 0.195 μV, so I set "Scale" to 0.195 to multiply our + data by 0.195. I also set the "Depth" property to F32 because a larger bit depth is required to correctly + represent the scaled values. + + Following a similar process as before for visualizing data, we can visualize the scaled data. + + +
+ Apply a filter to the signal. For the purpose of detecting spikes, I set a high-pass filter at 300 Hz. This is done + in your Bonsai workflow by connecting a `FrequencyFilter` operator to the second `ConvertScale` operator and setting + its "Cutoff1" property to 300. + + Following a similar process as before for visualizing data, we can visualize the scaled, filtered data. + + + Last, set a simple fixed threshold for detecting spikes. + + Following a similar process as before for visualizing data, we can visualize the spike data. + + + > [!TIP] + > You can test this spike detection workflow using a pre-recorded data known to have spikes. Simply recreate the + > data processing graph in a new workflow but replace the ephys data node (in the case of the headstage64, replace + > the `Rhd2164` node) with a `MatrixReader` that reads from the file containing spiky ephys data. + + + +[^1]: + +| Hardware | Configuration Operator | Amplifier Device | Ephys Data Operator | Data Frame | Shift | Scale | +|------------------------------|---------------------------------------------------------|-------------------------------------------------------------------------------------------------------------|-------------------------------------------|------------------------------------------------|--------|--------| +| Headstage64 | | [Intan Rhd2164](https://intantech.com/files/Intan_RHD2164_datasheet.pdf) | | | -32768 | 0.195 | +| HeadstageRhs2116 | | [Intan Rhs2116](https://intantech.com/files/Intan_RHS2116_datasheet.pdf) | | | -32768 | 0.195 | +| NeuropixelsV1eHeadstage | | [Neuropixels 1.0 probe](https://www.neuropixels.org/_files/ugd/328966_c5e4d31e8a974962b5eb8ec975408c9f.pdf) | | | -512 | * | +| NeuropixelsV2eHeadstage | | [Neuropixels 2.0 probe](https://www.neuropixels.org/_files/ugd/328966_2b39661f072d405b8d284c3c73588bc6.pdf) | | | -2048 | 2.4414 | + + +\* The Neuropixels 1.0 probes have selectable gain which has an effect on the multiplier for scaling the signal to μV. +Use the following formula to calculate the correct "Scale" value: $1.2e6/1024/gain$ \ No newline at end of file diff --git a/articles/tutorials/toc.yml b/articles/tutorials/toc.yml index ff9c189f..fc0663bb 100644 --- a/articles/tutorials/toc.yml +++ b/articles/tutorials/toc.yml @@ -1,2 +1,4 @@ - href: index.md - items: + items: [ + href: spikes.md + ] diff --git a/docfx.json b/docfx.json index 5eeb8477..fff474bf 100644 --- a/docfx.json +++ b/docfx.json @@ -62,7 +62,8 @@ "attributes", "customcontainers", "footnotes", - "figures" + "figures", + "emphasisextras" ], "alerts": { "IMPORTANT": "alert alert-warning" diff --git a/workflows/tutorials/spikes/configuration.bonsai b/workflows/tutorials/spikes/configuration.bonsai new file mode 100644 index 00000000..d723028d --- /dev/null +++ b/workflows/tutorials/spikes/configuration.bonsai @@ -0,0 +1,125 @@ + + + + + + + riffa + 0 + + + + + BreakoutBoard + + BreakoutBoard/Heartbeat + 0 + true + 10 + + + BreakoutBoard/AnalogIO + 6 + false + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + Input + Input + Input + Input + Input + Input + Input + Input + Input + Input + Input + Input + + + BreakoutBoard/DigitalIO + 7 + false + + + BreakoutBoard/OutputClock + 5 + false + 1000000 + 50 + 0 + + + BreakoutBoard/HarpSyncInput + 12 + false + Breakout + + + BreakoutBoard/MemoryMonitor + 10 + false + 10 + + + + + + Headstage64 + + Headstage64/Rhd2164 + 512 + true + Dsp146mHz + Low100mHz + High20000Hz + + + Headstage64/Bno055 + 513 + false + + + Headstage64/TS4231V1 + 514 + false + + + Headstage64/Headstage64ElectricalStimulator + 515 + + + Headstage64/Headstage64OpticalStimulator + 516 + + PortB + + + + + + 4096 + 2048 + + + + + + + + + + \ No newline at end of file diff --git a/workflows/tutorials/spikes/ephys-data.bonsai b/workflows/tutorials/spikes/ephys-data.bonsai new file mode 100644 index 00000000..bb261253 --- /dev/null +++ b/workflows/tutorials/spikes/ephys-data.bonsai @@ -0,0 +1,26 @@ + + + + + + + Headstage64/Rhd2164 + 30 + + + + Clock + + + AmplifierData + + + + + + + + \ No newline at end of file diff --git a/workflows/tutorials/spikes/spike-detection.bonsai b/workflows/tutorials/spikes/spike-detection.bonsai new file mode 100644 index 00000000..1c88131b --- /dev/null +++ b/workflows/tutorials/spikes/spike-detection.bonsai @@ -0,0 +1,75 @@ + + + + + + + Headstage64/Rhd2164 + 30 + + + + Clock + + + AmplifierData + + + + + 0 + 1 + 2 + 3 + + + + + + S16 + 1 + -32768 + + + + + F32 + 0.195 + 0 + + + + + 30000 + 300 + 0 + 60 + HighPass + + + + + 0 + 33 + + -50 + + AlignPeaks + + + + + + + + + + + + + + \ No newline at end of file diff --git a/workflows/tutorials/spikes/spikes.bonsai b/workflows/tutorials/spikes/spikes.bonsai new file mode 100644 index 00000000..498b43ce --- /dev/null +++ b/workflows/tutorials/spikes/spikes.bonsai @@ -0,0 +1,188 @@ + + + + + + + riffa + 0 + + + + + BreakoutBoard + + BreakoutBoard/Heartbeat + 0 + true + 10 + + + BreakoutBoard/AnalogIO + 6 + false + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + TenVolts + Input + Input + Input + Input + Input + Input + Input + Input + Input + Input + Input + Input + + + BreakoutBoard/DigitalIO + 7 + false + + + BreakoutBoard/OutputClock + 5 + false + 1000000 + 50 + 0 + + + BreakoutBoard/HarpSyncInput + 12 + false + Breakout + + + BreakoutBoard/MemoryMonitor + 10 + false + 10 + + + + + + Headstage64 + + Headstage64/Rhd2164 + 512 + true + Dsp146mHz + Low100mHz + High20000Hz + + + Headstage64/Bno055 + 513 + false + + + Headstage64/TS4231V1 + 514 + false + + + Headstage64/Headstage64ElectricalStimulator + 515 + + + Headstage64/Headstage64OpticalStimulator + 516 + + PortB + + + + + + 4096 + 2048 + + + + + Headstage64/Rhd2164 + 30 + + + + Clock + + + AmplifierData + + + + + 0 + 1 + 2 + 3 + + + + + + S16 + 1 + -32768 + + + + + F32 + 0.195 + 0 + + + + + 30000 + 300 + 0 + 60 + HighPass + + + + + 0 + 33 + + -50 + + AlignPeaks + + + + + + + + + + + + + + + + + \ No newline at end of file From f9f08c4f474545555570783e08fbf66ad816266c Mon Sep 17 00:00:00 2001 From: ChucklesOnGitHub Date: Mon, 4 Nov 2024 20:22:56 +0000 Subject: [PATCH 02/19] style proposal --- articles/tutorials/spikes.md | 150 +++++++++++++++++++---------------- 1 file changed, 81 insertions(+), 69 deletions(-) diff --git a/articles/tutorials/spikes.md b/articles/tutorials/spikes.md index ce29d51e..8197eb00 100644 --- a/articles/tutorials/spikes.md +++ b/articles/tutorials/spikes.md @@ -1,115 +1,128 @@ --- uid: spikes -title: Spike Detection +title: Working with ephys data in Bonsai + --- -This tutorial is dedicated to teaching how to use ONIX hardware and the OpenEphys.Onix1 Bonsai package to detect spikes -with a simple fixed threshold as well as visualize and save raw/spike data. The following workflow is an example of -what you can expect after finishing this tutorial by the end of the tutorial. +This tutorial shows how to use ONIX hardware and the OpenEphys.Onix1 Bonsai package to perform basic online signal processing +functions such as channel selection, filtering, visualization and a simple spike detection with using a fixed threshold. +It will guide you through building the following workflow. ::: workflow ![/workflows/tutorials/spikes.bonsai workflow](../../workflows/tutorials/spikes/spikes.bonsai) ::: > [!NOTE] -> Although this tutorial uses the headstage64 to exemplify the spike detection process, it's pretty similar for -> other hardware. This tutorial in addition to the [hardware guide](xref:hardware) for your particular -> ONIX headstage should be sufficient for you to be able to detect spikes with whatever you have your hands on. +> Although this tutorial uses headstage64, the process is similar for +> other ephys headstages. This tutorial assumes you are familiar with the [hardware guide](xref:hardware) of the +> ONIX headstage you intend to use. Use the table at the bottom of this tutorial[^1] as a reference for which ephys you need +> and scaling corresponds to each headstage and links to relevant documentation. 1. [Get started](xref:getting-started) in Bonsai. In particular, - [download the necessary Bonsai packages](xref:install-configure-bonsai#install-packages-in-bonsai). - If you've already downloaded them, - [check for updates](xref:install-configure-bonsai#update-packages-in-bonsai). This tutorial assumes you're using the + [download the necessary Bonsai packages](xref:install-configure-bonsai#install-packages-in-bonsai) or + [check for updates](xref:install-configure-bonsai#update-packages-in-bonsai) to make sure you have the latest version. This tutorial assumes you're using the latest software. 1. Create the top-level configuration graph - + ::: workflow ![/workflows/tutorials/spikes/configuration.bonsai workflow](../../workflows/tutorials/spikes/configuration.bonsai) ::: - In this workflow, all devices on the breakout board and all devices on the headstage64 except the Rhd2164 are - configured to be disabled at run-time. For the sole purpose of teaching how to detect spikes, I only need the data - from Rhd2164 Intan amplifier and don't have to bother with all that other stuff. - - If you are collecting electrophysiology data from hardware that is not the headstage64, replace the - node with a [configuration operator](xref:configure) that corresponds to - your hardware, and confirm the device that streams electrophysiology data on that headstage is enabled. + 11. Place the [configuration operator](xref:configure) that corresponds to + the headstage you intend to use. In our example, this is . -1. Place the relevant operator to stream electrophysiology data from your hardware and select the - relevant output members + 11. Make sure the device that streams electrophysiology data is enabled. We will only be using the Rhd2164 device on headstage64 (the Intan amplifier), so you could disable other devices on the headstage or on the breakout board. + +1. Place the relevant operator to stream electrophysiology data from your headstage and select the relevant output members. ::: workflow ![/workflows/tutorials/spikes/ephys-data.bonsai workflow](../../workflows/tutorials/spikes/ephys-data.bonsai) ::: - Because I am using headstage64 in this example and the headstage64 streams electrophysiology through Rhd2164 Intan - amplifier, I place the node onto the workflow. If you are using different - hardware, the table at the bottom of this tutorial[^1] provides a reference for which ephys you need - to place onto your workflow for your particular hardware and links to relevant documentation. Select the relevant - members from the frames that are produced by the chosen ephys data operators. In this case, those members are - "AmplifierData" and "Clock". This is most easily performed by clicking one of the members that appears after - hovering over the output option in the context menu that appears after right-clicking the `Rhd2164` node. + 11. The device on headstage64 that streams electrophysiology data is the Rhd2164 Intan + amplifier, so we need to place the node onto the workflow. + + 11. Select the relevant members from the data frames that the data operator produces. In this example, the relevant members are + "AmplifierData" and "Clock". To do this, right-click the `Rhd2164` node, hover over the output option in the context menu and select one of the members from the list. - Let's visualize the raw data. Start the workflow by pressing F5 or clicking the green triangle play - button at the top of the workflow editor. Double click the selected member that is streaming ephys data. In my case, - that's the `MemberSelector` selecting "AmplifierData". +1. Check that you can stream data by visualizing it. + + 11. Start the workflow by clicking the green triangle play + button at the top of the workflow editor or pressing F5. + + 11. Double click the selected member that is streaming ephys data. In this example, + that's the `MemberSelector` that selects "AmplifierData". + + For ease of visualization, click on the visualizer and configure it: + - Increase the History length to 100 + - -1. Create the data processing graph for spike detection +1. Create the data processing graph for processing the ephys data. ::: workflow ![/workflows/tutorials/spikes/spike-detection.bonsai workflow](../../workflows/tutorials/spikes/spike-detection.bonsai) ::: - First, select from which channels you want to detect spikes by connecting a `SelectChannels` node to your - electrophysiology data stream and editing its "Channels" property. + 11. Select and reorder channels of interest. + + Connect a `SelectChannels` node to the electrophysiology data stream and edit its "Channels" property. + Remember indexing starts at 0. Use commas to separate multiple channels and brackets for ranges. + Reorder channels by writing the channel numbers in the order in which you want to visualize the channels. - Center the dynamic range of the signal around zero and scale the ADC signal to μV. Although the following - calculation can be done in one `ConvertScale` operator, the calculations are more straightforward using two of them - connected in series because the `ConvertScale` operator applies the "Shift" offset after applying the "Scale" scalar - instead of the reverse order of operation. Connect the first `ConvertScale` operator to the `SelectChannels` operator. - - Center the dynamic range of the signal around zero by subtracting 2^bit depth - 1^ from the signal. This is done - in your Bonsai workflow by editing the "Shift" property of the first `ConvertScale` operator. If you are unsure - about the bit depth of the electrophysiology data from your particular hardware, refer to the table at the - bottom of this tutorial.[^1] The headstage64's Rhd2164 device has a bit depth of 16, so I set "Shift" to 32768 to - subtract 32768 from our data. I also set the "Depth" property to S16 so that the negative data is properly - represented. S16 refers to a signed 16-bit data type - - Scale the ADC signal to μV by multiplying all values in the ADC signal by a scalar that is determined by the - gain of the amplifier and resolution the ADC contained in the amplifier device This is done in your Bonsai - workflow by editing the "Scale" property of the second `ConvertScale` operator. If you are unsure about the step - size of the electrophysiology data from your particular hardware, refer to the table at the bottom of this - tutorial.[^1] The Rhd2164 amplifier device has a step size of 0.195 μV, so I set "Scale" to 0.195 to multiply our - data by 0.195. I also set the "Depth" property to F32 because a larger bit depth is required to correctly - represent the scaled values. - - Following a similar process as before for visualizing data, we can visualize the scaled data. + 11. Center the dynamic range of the signal around zero + + Connect a `ConvertScale` operator to the `SelectChannels` operator. + - Edit its "Shift" property to subtract 2^bit depth - 1^ from the signal. Refer to the table at the bottom of this tutorial to find the Shift necessary for each device.[^1] + In this example, the Rhd2164 device on headstage64 has a bit depth of 16, so "Shift" has to be set to 32768 to subtract 32768 from the raw data. + - Set the "Depth" property to F32 because this bit depth is required to correctly represent scaled data from all devices. + + 11. Scale the ADC signal to microvolts. + + Connect a second `ConvertScale` operator to the `ConvertScale` operator that is already on the workflow. + - Edit its "Scale" property to multiply the signal by a scalar in order to get microvolt values. This scalar is determined by the gain of the amplifier and resolution the ADC contained in the amplifier device. Refer to the table at the bottom of this tutorial to find the Scale necessary for each device.[^1] + In this example, the Rhd2164 device on headstage64 has a step size of 0.195 μV, so "Scale" has to be set to 0.195 to multiply the centered data by 0.195. + - Keep the "Depth" property at F32. + + > [!NOTE] + > Although both the Shift and Scale calculation can be done in one `ConvertScale` operator, the calculations are more straightforward using two operators + > connected in series because the `ConvertScale` operator applies the "Shift" offset after applying the "Scale" scalar so if we used a single operator, + > we would have to scale the Shift parameter. + +1. Visualize the centered and scaled data. + -
- Apply a filter to the signal. For the purpose of detecting spikes, I set a high-pass filter at 300 Hz. This is done - in your Bonsai workflow by connecting a `FrequencyFilter` operator to the second `ConvertScale` operator and setting - its "Cutoff1" property to 300. +1. Apply a frequency filter to the signal. + + Connect a `FrequencyFilter` operator to the second `ConvertScale` operator and set its properties. + The sample rate for ephys data in all devices is 30kHz. + For looking at spikes, set the "Cutoff1" property to 300 to apply a high-pass filter at 300 Hz. + +1. Visualize the scaled, filtered data. - Following a similar process as before for visualizing data, we can visualize the scaled, filtered data. - Last, set a simple fixed threshold for detecting spikes. +1. Detect spikes + + Based on the amplitude of the signal, set a fixed threshold for detecting spikes. + +1. Visualize the spike data. - Following a similar process as before for visualizing data, we can visualize the spike data. > [!TIP] > You can test this spike detection workflow using a pre-recorded data known to have spikes. Simply recreate the - > data processing graph in a new workflow but replace the ephys data node (in the case of the headstage64, replace - > the `Rhd2164` node) with a `MatrixReader` that reads from the file containing spiky ephys data. + > data processing graph in a new workflow and replace the ephys data node (in the case of the headstage64, replace + > the `Rhd2164` node) with a `MatrixReader` that reads from the file containing spiking ephys data. This tutorial shows how to use ONIX hardware and the OpenEphys.Onix1 Bonsai package to perform basic online signal -processing functions such as channel selection, filtering, visualization and a simple spike detection using a fixed -threshold. It will guide you through building the following workflow. +processing functions in Bonsai such as channel selection and reordering, filtering and event detection such as a fixed +threshold crossing. + +This type of processing is important for real-time feedback for monitoring data being acquired. However, Bonsai's integrated visualizers and event detectors are limited. +We recommend using the tools available in the Open Ephys GUI for specialized ephys visualizations, in particular, for data from very dense arrays such as Neuropixels probes. + + +More advanced event detection algorithms such as spike sorting, ripple detection, etc. need specific implementations in Bonsai. Event detection in Bonsai will be faster and it allows actuation using ONIX or other hardware for closed-loop applications. + +This tutorial will guide you through building the following workflow: ::: workflow ![/workflows/tutorials/spikes.bonsai workflow](../../workflows/tutorials/spikes/spikes.bonsai) From 8b3f2c32cd88d861d9d766b84f83987a6c049201 Mon Sep 17 00:00:00 2001 From: ChucklesOnGitHub Date: Tue, 12 Nov 2024 11:50:11 +0000 Subject: [PATCH 05/19] heading and title change --- articles/tutorials/spikes.md | 230 ++++++++++++++++++----------------- 1 file changed, 118 insertions(+), 112 deletions(-) diff --git a/articles/tutorials/spikes.md b/articles/tutorials/spikes.md index 1927cf1a..0050a068 100644 --- a/articles/tutorials/spikes.md +++ b/articles/tutorials/spikes.md @@ -1,6 +1,6 @@ --- uid: spikes -title: Signal processing of ephys data in Bonsai +title: Processing ephys data in Bonsai --- @@ -28,117 +28,123 @@ This tutorial will guide you through building the following workflow: > Use the table at the bottom of this tutorial[^1] as a reference for which ephys you need and scaling > corresponds to each headstage and links to relevant documentation. -1. [Get started](xref:getting-started) in Bonsai. In particular, - [download the necessary Bonsai packages](xref:install-configure-bonsai#install-packages-in-bonsai) or - [check for updates](xref:install-configure-bonsai#update-packages-in-bonsai). This tutorial assumes - you're using the latest software. - -1. Configure the hardware. - - ::: workflow - ![/workflows/tutorials/spikes/configuration.bonsai workflow](../../workflows/tutorials/spikes/configuration.bonsai) - ::: - - - This is accomplished by constructing a [top-level configuration chain](xref:initialize-onicontext). First, place the - [configuration operator](xref:configure) that corresponds to the hardware you intend to use between - and . In our example, this is - and . Confirm the device - that streams electrophysiology data is enabled. The Rhd2164 device (an Intan amplifier) on the headstage64 is the - only device used in this tutorial, so you could disable other devices on the headstage and on the breakout board. - -1. Stream ephys data into Bonsai. - - ::: workflow - ![/workflows/tutorials/spikes/ephys-data.bonsai workflow](../../workflows/tutorials/spikes/ephys-data.bonsai) - ::: - - Put the relevant operator to stream electrophysiology data from your headstage and select the relevant output - members. Because the device on headstage64 that streams electrophysiology data is the Rhd2164 Intan amplifier, we - placed the node onto the workflow. Select the relevant members from the data - frames that the data operator produces. In this example, the relevant members are "AmplifierData" and "Clock". To - those members, we right-click the `Rhd2164` node, hover over the output option in the context menu, and select it from - the list. - - - - [Visualize the raw data](xref:visualize-data) to confirm the ephys data operator is streaming data. - - - - - -1. Select channels, and shift/scale ephys data from those channels. - - ::: workflow - ![/workflows/tutorials/spikes/select-convert-ephys-data.bonsai workflow](../../workflows/tutorials/spikes/select-convert-ephys-data.bonsai) - ::: - - 1. Select and reorder channels of interest. - - Connect a operator to the electrophysiology data stream and edit its "Channels" property. - Remember indexing starts at 0. Use commas to separate multiple channels and brackets for ranges. - Reorder channels by writing the channel numbers in the order in which you want to visualize the channels. - - 1. Center the dynamic range of the ADC signal around zero - - Connect a operator to the `SelectChannels` operator and set its properties: - - Edit its "Shift" property to subtract 2^bit depth - 1^ from the signal. Refer to the table at the bottom of - this tutorial to find the Shift necessary for each device.[^1] In this example, we "Shift" -32768 because the - Rhd2164 device outputs 16-bit data. - - Set the "Depth" property to F32 because this bit depth is required to correctly represent scaled data from all - devices. - - 1. Scale the ADC signal to microvolts. - - Connect a second `ConvertScale` operator to the first `ConvertScale` operator and set its properties: - - Edit its "Scale" property to multiply the signal by a scalar in order to get microvolt values. This scalar is - determined by the gain of the amplifier and resolution the ADC contained in the amplifier device. Refer to the - table at the bottom of this tutorial to find the "Scale" necessary for each device.[^1] In this example, we - "Scale" by 0.195 because the Rhd2164 device on headstage64 has a step size of 0.195 μV/bit - - Keep the "Depth" property at F32. - - Visualize the transformed data to confirm the output of the shifting and scaling operations - comport with expectations. - - > [!NOTE] - > Although both the Shift and Scale calculation can be done in one `ConvertScale` operator, the calculations are - > more straightforward using two operators connected in series because the `ConvertScale` operator applies the - > "Shift" offset after applying the "Scale" scalar so if we used a single operator, we would have to scale the Shift - > parameter. - - - -1. Filter ephys data. - - ::: workflow - ![/workflows/tutorials/spikes/filter-ephys-data.bonsai workflow](../../workflows/tutorials/spikes/filter-ephys-data.bonsai) - ::: - - Connect a `FrequencyFilter` operator to the second `ConvertScale` operator and set its properties. - - Set its "SampleRate" property to 30000. Ephys data in all devices is 30 kHz. - - Set the "Cutoff1" property to an adequate value for looking at spikes. In this examples, we use 300 Hz as the - lower cutoff frequency for a high-pass filter. - - - -1. Detect spikes. - - ::: workflow - ![/workflows/tutorials/spikes/spikes.bonsai workflow](../../workflows/tutorials/spikes/spikes.bonsai) - ::: - - Based on the amplitude of the signal, set a fixed threshold for detecting spikes. - - Visualize the spike data. - - - - > [!TIP] - > You can test this spike detection workflow using a pre-recorded data known to have spikes. Simply recreate the - > data processing graph in a new workflow and replace the ephys data node (in the case of the headstage64, replace - > the `Rhd2164` node) with a `MatrixReader` that reads from the file containing spiking ephys data. - +## Set up Bonsai + +Follow the [Getting Started](xref:getting-started) guide to set up Bonsai. In particular, +[download the necessary Bonsai packages](xref:install-configure-bonsai#install-packages-in-bonsai) or +[check for updates](xref:install-configure-bonsai#update-packages-in-bonsai). This tutorial assumes +you're using the latest software. + +## Configure the hardware + +::: workflow +![/workflows/tutorials/spikes/configuration.bonsai workflow](../../workflows/tutorials/spikes/configuration.bonsai) +::: + + +This is accomplished by constructing a [top-level configuration chain](xref:initialize-onicontext). First, place the +[configuration operator](xref:configure) that corresponds to the hardware you intend to use between + and . In our example, this is + and . Confirm the device +that streams electrophysiology data is enabled. The Rhd2164 device (an Intan amplifier) on the headstage64 is the +only device used in this tutorial, so you could disable other devices on the headstage and on the breakout board. + +## Stream ephys data into Bonsai + +::: workflow +![/workflows/tutorials/spikes/ephys-data.bonsai workflow](../../workflows/tutorials/spikes/ephys-data.bonsai) +::: + +Put the relevant operator to stream electrophysiology data from your headstage and select the relevant output +members. Because the device on headstage64 that streams electrophysiology data is the Rhd2164 Intan amplifier, we +placed the node onto the workflow. Select the relevant members from the data +frames that the data operator produces. In this example, the relevant members are "AmplifierData" and "Clock". To +those members, we right-click the `Rhd2164` node, hover over the output option in the context menu, and select it from +the list. + + + +[Visualize the raw data](xref:visualize-data) to confirm the ephys data operator is streaming data. + + + + + +## Select and reorder channels + +::: workflow +![/workflows/tutorials/spikes/select-convert-ephys-data.bonsai workflow](../../workflows/tutorials/spikes/select-convert-ephys-data.bonsai) +::: + +Connect a operator to the electrophysiology data stream and edit its "Channels" property. +Remember indexing starts at 0. Use commas to separate multiple channels and brackets for ranges. +Reorder channels by writing the channel numbers in the order in which you want to visualize the channels. + +## Convert data to physical units + +::: workflow +![/workflows/tutorials/spikes/select-convert-ephys-data.bonsai workflow](../../workflows/tutorials/spikes/select-convert-ephys-data.bonsai) +::: + +1. Center the dynamic range of the ADC signal around zero + + Connect a operator to the `SelectChannels` operator and set its properties: + - Edit its "Shift" property to subtract 2^bit depth - 1^ from the signal. Refer to the table at the bottom of + this tutorial to find the Shift necessary for each device.[^1] In this example, we "Shift" -32768 because the + Rhd2164 device outputs 16-bit data. + - Set the "Depth" property to F32 because this bit depth is required to correctly represent scaled data from all + devices. + +1. Scale the ADC signal to microvolts. + + Connect a second `ConvertScale` operator to the first `ConvertScale` operator and set its properties: + - Edit its "Scale" property to multiply the signal by a scalar in order to get microvolt values. This scalar is + determined by the gain of the amplifier and resolution the ADC contained in the amplifier device. Refer to the + table at the bottom of this tutorial to find the "Scale" necessary for each device.[^1] In this example, we + "Scale" by 0.195 because the Rhd2164 device on headstage64 has a step size of 0.195 μV/bit + - Keep the "Depth" property at F32. + +Visualize the transformed data to confirm the output of the shifting and scaling operations +comport with expectations. + +> [!NOTE] +> Although both the Shift and Scale calculation can be done in one `ConvertScale` operator, the calculations are +> more straightforward using two operators connected in series because the `ConvertScale` operator applies the +> "Shift" offset after applying the "Scale" scalar so if we used a single operator, we would have to scale the Shift +> parameter. + + + +## Apply a frequency filter + +::: workflow +![/workflows/tutorials/spikes/filter-ephys-data.bonsai workflow](../../workflows/tutorials/spikes/filter-ephys-data.bonsai) +::: + +Connect a `FrequencyFilter` operator to the second `ConvertScale` operator and set its properties. +- Set its "SampleRate" property to 30000. Ephys data in all devices is 30 kHz. +- Set the "Cutoff1" property to an adequate value for looking at spikes. In this examples, we use 300 Hz as the + lower cutoff frequency for a high-pass filter. + + + +## Detect events + +::: workflow +![/workflows/tutorials/spikes/spikes.bonsai workflow](../../workflows/tutorials/spikes/spikes.bonsai) +::: + +Based on the amplitude of the signal, set a fixed threshold for detecting spikes. + +Visualize the spike data. + + + +> [!TIP] +> You can test this spike detection workflow using a pre-recorded data known to have spikes. Simply recreate the +> data processing graph in a new workflow and replace the ephys data node (in the case of the headstage64, replace +> the `Rhd2164` node) with a `MatrixReader` that reads from the file containing spiking ephys data. + -[Visualize the raw data](xref:visualize-data) to confirm the ephys data operator is streaming data. +Visualize the raw data to confirm the ephys data operator is streaming data. From e1bb7ad4ab14d88643b91eae1d609ddf30bc153e Mon Sep 17 00:00:00 2001 From: ChucklesOnGitHub Date: Tue, 12 Nov 2024 13:17:29 +0000 Subject: [PATCH 11/19] content edits to be more specific --- articles/tutorials/spikes.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/articles/tutorials/spikes.md b/articles/tutorials/spikes.md index 80a1fa05..d1c097db 100644 --- a/articles/tutorials/spikes.md +++ b/articles/tutorials/spikes.md @@ -10,8 +10,7 @@ This tutorial shows how to use ONIX hardware and the OpenEphys.Onix1 Bonsai pack processing functions in Bonsai such as channel selection and reordering, filtering and event detection such as a fixed threshold crossing. -This type of processing is important for real-time feedback for monitoring data being acquired. However, Bonsai's integrated visualizers and event detectors are limited. -We recommend using the tools available in the Open Ephys GUI for specialized ephys visualizations, in particular, for data from very dense arrays such as Neuropixels probes. +This type of processing is important for real-time feedback while monitoring data being acquired. However, Bonsai's integrated visualizers and event detectors are limited, so we recommend using the tools available in the Open Ephys GUI for specialized ephys visualizations. The Open Ephys GUI is particularly suited for visualizing data from very dense arrays such as Neuropixels probes. More advanced event detection algorithms such as spike sorting, ripple detection, etc. need specific implementations in Bonsai. Event detection in Bonsai will be faster and it allows actuation using ONIX or other hardware for closed-loop applications. @@ -44,7 +43,7 @@ you're using the latest software. ![/workflows/tutorials/spikes/configuration.bonsai workflow](../../workflows/tutorials/spikes/configuration.bonsai) ::: -This is accomplished by constructing a [top-level configuration chain](xref:initialize-onicontext). First, place the +This is accomplished by constructing a [top-level configuration chain](xref:initialize-onicontext): place the [configuration operator](xref:configure) that corresponds to the hardware you intend to use between and . In our example, this is and . Confirm the device @@ -57,11 +56,10 @@ only device used in this tutorial, so you could disable other devices on the hea ![/workflows/tutorials/spikes/ephys-data.bonsai workflow](../../workflows/tutorials/spikes/ephys-data.bonsai) ::: -Put the relevant operator to stream electrophysiology data from your headstage and select the relevant output +Place the relevant operator to stream electrophysiology data from your headstage and select the relevant output members. Because the device on headstage64 that streams electrophysiology data is the Rhd2164 Intan amplifier, we placed the node onto the workflow. Select the relevant members from the data -frames that the data operator produces. In this example, the relevant members are "AmplifierData" and "Clock". To -those members, we right-click the `Rhd2164` node, hover over the output option in the context menu, and select it from +frames that the data operator produces. In this example, the relevant members are "AmplifierData" and "Clock". To select those members, right-click the `Rhd2164` node, hover over the output option in the context menu, and select it from the list. @@ -124,19 +122,26 @@ comport with expectations. ::: Connect a `FrequencyFilter` operator to the second `ConvertScale` operator and set its properties. -- Set its "SampleRate" property to 30000. Ephys data in all devices is 30 kHz. -- Set the "Cutoff1" property to an adequate value for looking at spikes. In this examples, we use 300 Hz as the - lower cutoff frequency for a high-pass filter. +- Set its "SampleRate" property to 30000. Ephys data in all devices is 30 kHz. +- Set the "FilterType" property to an adequate type. In this example, we use a high pass filter to look at spikes. +- Set the "Cutoff1" and "Cutoff2" properties to an adequate value. In this examples, we use 300 Hz as the + lower cutoff frequency. +> [!TIP] +> If you choose to save data, make sure you place the `MatrixWriter` operator before filtering and scaling to save raw +> data instead of scaled or scaled, filtered data. The `FrequencyFilter` operator could remove signals from a bandwidth +> of interest and the second `ConvertScale` value increase the size of your data without increasing meaningful +> information. + ## Detect events ::: workflow ![/workflows/tutorials/spikes/spikes.bonsai workflow](../../workflows/tutorials/spikes/spikes.bonsai) ::: -Based on the amplitude of the signal, set a fixed threshold for detecting spikes. +Based on the amplitude of the signal on the selected channel, set a fixed threshold for detecting spikes. Visualize the spike data. @@ -147,11 +152,6 @@ Visualize the spike data. > data processing graph in a new workflow and replace the ephys data node (in the case of the headstage64, replace > the `Rhd2164` node) with a `MatrixReader` that reads from the file containing spiking ephys data. - [^1]: From 8c163feda997eac3b7b13d27b47452926bdaabbe Mon Sep 17 00:00:00 2001 From: ChucklesOnGitHub Date: Tue, 12 Nov 2024 13:34:52 +0000 Subject: [PATCH 12/19] clarity edits --- articles/tutorials/spikes.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/articles/tutorials/spikes.md b/articles/tutorials/spikes.md index d1c097db..22d0d3e7 100644 --- a/articles/tutorials/spikes.md +++ b/articles/tutorials/spikes.md @@ -43,10 +43,10 @@ you're using the latest software. ![/workflows/tutorials/spikes/configuration.bonsai workflow](../../workflows/tutorials/spikes/configuration.bonsai) ::: -This is accomplished by constructing a [top-level configuration chain](xref:initialize-onicontext): place the -[configuration operator](xref:configure) that corresponds to the hardware you intend to use between - and . In our example, this is - and . Confirm the device +Construct a [top-level configuration chain](xref:initialize-onicontext): place the +[configuration operators](xref:configure) that correspond to the hardware you intend to use between + and . In this example, this is + and . Confirm that the device that streams electrophysiology data is enabled. The Rhd2164 device (an Intan amplifier) on the headstage64 is the only device used in this tutorial, so you could disable other devices on the headstage and on the breakout board. @@ -91,7 +91,7 @@ Reorder channels by writing the channel numbers in the order in which you want t Connect a operator to the `SelectChannels` operator and set its properties: - Edit its "Shift" property to subtract 2^bit depth - 1^ from the signal. Refer to the table at the bottom of this tutorial to find the Shift necessary for each device.[^1] In this example, we "Shift" -32768 because the - Rhd2164 device outputs 16-bit data. + Rhd2164 device outputs unsigned 16-bit data. - Set the "Depth" property to F32 because this bit depth is required to correctly represent scaled data from all devices. @@ -105,7 +105,7 @@ Reorder channels by writing the channel numbers in the order in which you want t - Keep the "Depth" property at F32. Visualize the transformed data to confirm the output of the shifting and scaling operations -comport with expectations. +worked as expected, i.e. that the signal is centered around zero and that the values make sense in microvolts. > [!NOTE] > Although both the Shift and Scale calculation can be done in one `ConvertScale` operator, the calculations are @@ -148,9 +148,9 @@ Visualize the spike data. > [!TIP] -> You can test this spike detection workflow using a pre-recorded data known to have spikes. Simply recreate the -> data processing graph in a new workflow and replace the ephys data node (in the case of the headstage64, replace -> the `Rhd2164` node) with a `MatrixReader` that reads from the file containing spiking ephys data. +> You can test the spike detection using a pre-recorded data known to have spikes: recreate the +> workflow from this example without the hardware configuration chain in a new workflow and replace the ephys data node (in the case of the headstage64, replace +> the `Rhd2164` node) with a `MatrixReader` that reads from the file containing spiking ephys data in unsigned 16-bit format. [^1]: From a120d572e6d2509cfbc77453cf5f1f91593ebd69 Mon Sep 17 00:00:00 2001 From: ChucklesOnGitHub Date: Tue, 12 Nov 2024 14:26:46 +0000 Subject: [PATCH 13/19] clarity updates --- articles/getting-started/visualize-data.md | 31 +++++++++++----------- articles/tutorials/spikes.md | 27 +++++++++---------- 2 files changed, 27 insertions(+), 31 deletions(-) diff --git a/articles/getting-started/visualize-data.md b/articles/getting-started/visualize-data.md index 24f453a3..f6ff4174 100644 --- a/articles/getting-started/visualize-data.md +++ b/articles/getting-started/visualize-data.md @@ -3,15 +3,15 @@ uid: visualize-data title: Visualize Data --- -Bonsai includes default ["type visualizers"](https://bonsai-rx.org/docs/articles/editor.html?#type-visualizers) that open as individual windows when double-clicking a data node to display data produced by an operator while the workflow is running. +Bonsai has ["type visualizers"](https://bonsai-rx.org/docs/articles/editor.html?#type-visualizers) that open as individual windows when double-clicking a data node to display data produced by an operator while the workflow is running. -Not all nodes have type visualizers, because the operator data output type has to be compatible with Bonsai's default visualizers. Data frames from OpenEphys.Onix1, for example, aren't compatible with Bonsai's default type visualizers, but its data members are. +Not all nodes have type visualizers, because the operator data output type has to be compatible with Bonsai's default type visualizers. Data frames from the OpenEphys.Onix1 library, for example, aren't compatible with Bonsai's default type visualizers, but its data members are. After performing member selection on operators from the OpenEphys.Onix1 library, you can use the default type visualizers. -Other visualizers are available as standalone operators and can be found in the `Bonsai.Design.Visualizers` package, which can be installed in the Bonsai package manager. These visualizer operators must be placed in the workflow, linked to a data operator. The visualizer can be opened by double-clicking the visualizer node instead of the preceding data node. +Other visualizers for use with other libraries are available as standalone operators and can be found in the `Bonsai.Design.Visualizers` package, which can be installed in the Bonsai package manager. These visualizer operators must be placed in the workflow and linked to a data operator. The visualizer window can be opened by double-clicking the visualizer node instead of the preceding data node. ## Selecting operator data members for visualization -For nodes that require it such as ONIX [data I/O operators](xref:dataio), select the desired member: +If the node whose data you want to visualize requires member selection, such as ONIX [data I/O operators](xref:dataio): 1. Right-click the node that corresponds to the data I/O operator you'd like to visualize. 1. Hover over the "Output" option that appears in the context menu. 1. Click the member you would like to visualize from the list of members. @@ -42,21 +42,20 @@ Open the visualizer and check: like to visualize. 1. Visualize the data. - > [!NOTE] - > Data will only be visualized if the operator is producing data. If you can't see any data, check that: - > - The device from which you are trying to read is enabled. - > - Events are occurring. Some devices are stream-based and some are device-based. Event-based devices only produce data upon certain - > events. For example, the operator only produces data when the digital - > port status changes state. + > [!NOTE] + > Data will only be visualized if the operator is producing data. If you can't see any data, check that: + > - The device from which you are trying to read is enabled. + > - Events are occurring. Some devices are stream-based and some are device-based. Event-based devices only produce data upon certain + > events. For example, the operator only produces data when the digital + > port status changes state. - > [!TIP] - > Visualizers can be changed while the workflow is running, so Selecting and Opening visualizers steps can be done in any order. This allows you to try different visualizers (one at a time), which is particularly helpful if you don't know which visualizer you want to use. -## Configuring visualizers -Some visualizers, in particular, those that involve plots, allow additional - configuration. + > [!TIP] + > Visualizers can be changed while the workflow is running, so Selecting and Opening visualizers steps can be done in any order. This allows you to try different visualizers (one at a time), which is particularly helpful if you don't know which visualizer you want to use. - 1. Right-click the visualizer window to gain access to configuration options. +## Configuring visualizers +Some visualizers, in particular those that involve plots, allow additional configuration. +Right-click the visualizer window to gain access to configuration options. For example, the MatVisualizer allows configuration of: - X and Y scale: click to toggle between "auto" and fixed values. diff --git a/articles/tutorials/spikes.md b/articles/tutorials/spikes.md index 22d0d3e7..1ab550fd 100644 --- a/articles/tutorials/spikes.md +++ b/articles/tutorials/spikes.md @@ -7,13 +7,12 @@ title: Processing ephys data in Bonsai This tutorial shows how to use ONIX hardware and the OpenEphys.Onix1 Bonsai package to perform basic online signal -processing functions in Bonsai such as channel selection and reordering, filtering and event detection such as a fixed -threshold crossing. +on electrophysiology data in Bonsai such as channel selection and reordering, frequency filtering and event detection (in this example, spike detection using a fixed threshold crossing). -This type of processing is important for real-time feedback while monitoring data being acquired. However, Bonsai's integrated visualizers and event detectors are limited, so we recommend using the tools available in the Open Ephys GUI for specialized ephys visualizations. The Open Ephys GUI is particularly suited for visualizing data from very dense arrays such as Neuropixels probes. +This type of processing is important for real-time feedback while monitoring data during acquisition. However, Bonsai's integrated visualizers and event detectors are limited, so we recommend using the tools available in the Open Ephys GUI for specialized ephys visualizations. The Open Ephys GUI is particularly suited for visualizing data from very dense arrays such as Neuropixels probes. -More advanced event detection algorithms such as spike sorting, ripple detection, etc. need specific implementations in Bonsai. Event detection in Bonsai will be faster and it allows actuation using ONIX or other hardware for closed-loop applications. + Event detection in Bonsai will be faster and it allows actuation using ONIX or other hardware for closed-loop applications. However, more advanced event detection algorithms such as spike sorting, ripple detection, etc. need specific implementations in Bonsai. This tutorial will guide you through building the following workflow: @@ -24,10 +23,10 @@ This tutorial will guide you through building the following workflow: > [!NOTE] > Although this tutorial uses headstage64 as an example, the process is similar for other ephys headstages. This > tutorial assumes you are familiar with the [hardware guide](xref:hardware) of the ONIX headstage you intend to use. -> Use the table at the bottom of this tutorial[^1] as a reference for which ephys you need and scaling -> corresponds to each headstage and links to relevant documentation. +> Use the table at the bottom of this tutorial[^1] as a reference for which ephys and scaling +> you need to use for each headstage, and links to relevant documentation. -## Set up and get to know Bonsai +## Set up and get started in Bonsai Follow the [Getting Started](xref:getting-started) guide to set up and get familiarized with Bonsai. In particular: @@ -45,10 +44,10 @@ you're using the latest software. Construct a [top-level configuration chain](xref:initialize-onicontext): place the [configuration operators](xref:configure) that correspond to the hardware you intend to use between - and . In this example, this is + and . In this example, these are and . Confirm that the device that streams electrophysiology data is enabled. The Rhd2164 device (an Intan amplifier) on the headstage64 is the -only device used in this tutorial, so you could disable other devices on the headstage and on the breakout board. +only device used in this tutorial, so you could disable other devices on the headstage and on the breakout board to improve performance if you wanted to. ## Stream ephys data into Bonsai @@ -77,7 +76,7 @@ Visualize the raw data to confirm the ephys data operator is streaming data. ::: Connect a operator to the electrophysiology data stream and edit its "Channels" property. -Remember indexing starts at 0. Use commas to separate multiple channels and brackets for ranges. +Remember indexing in Bonsai starts at 0. Use commas to separate multiple channels and brackets for ranges. Reorder channels by writing the channel numbers in the order in which you want to visualize the channels. ## Convert data to physical units @@ -124,16 +123,14 @@ worked as expected, i.e. that the signal is centered around zero and that the va Connect a `FrequencyFilter` operator to the second `ConvertScale` operator and set its properties. - Set its "SampleRate" property to 30000. Ephys data in all devices is 30 kHz. - Set the "FilterType" property to an adequate type. In this example, we use a high pass filter to look at spikes. -- Set the "Cutoff1" and "Cutoff2" properties to an adequate value. In this examples, we use 300 Hz as the +- Set the "Cutoff1" and "Cutoff2" properties to an adequate value. In this example, we use 300 Hz as the lower cutoff frequency. > [!TIP] -> If you choose to save data, make sure you place the `MatrixWriter` operator before filtering and scaling to save raw -> data instead of scaled or scaled, filtered data. The `FrequencyFilter` operator could remove signals from a bandwidth -> of interest and the second `ConvertScale` value increase the size of your data without increasing meaningful -> information. +> If you choose to save data, we recommend you place the `MatrixWriter` operator before filtering and scaling to save raw +> data instead of scaled or filtered data. Filtering with the `FrequencyFilter` operator before recording could remove signals from a bandwidth of interest and converting to microvolts with the second `ConvertScale` operator could increase the size of your data without increasing meaningful information. ## Detect events From 0078b5e97fbcaf7a43896969da76d7a091b43bfd Mon Sep 17 00:00:00 2001 From: cjsha Date: Tue, 12 Nov 2024 12:29:42 -0500 Subject: [PATCH 14/19] Add workflows for tutorial changes Also: - Change tutorial title - Rename files - Change heading size --- .../{spikes.md => basic-ephys-processing.md} | 19 ++++--- articles/tutorials/toc.yml | 2 +- template/public/main.css | 10 +--- .../configuration.bonsai | 0 .../convert-units.bonsai} | 0 .../ephys-data.bonsai | 0 .../filter-ephys-data.bonsai | 0 .../select-convert-ephys-data.bonsai | 54 +++++++++++++++++++ .../select-reorder-channels.bonsai | 38 +++++++++++++ .../spike-detection.bonsai | 0 .../spikes.bonsai | 0 11 files changed, 104 insertions(+), 19 deletions(-) rename articles/tutorials/{spikes.md => basic-ephys-processing.md} (90%) rename workflows/tutorials/{spikes => basic-ephys-processing}/configuration.bonsai (100%) rename workflows/tutorials/{spikes/select-convert-ephys-data.bonsai => basic-ephys-processing/convert-units.bonsai} (100%) rename workflows/tutorials/{spikes => basic-ephys-processing}/ephys-data.bonsai (100%) rename workflows/tutorials/{spikes => basic-ephys-processing}/filter-ephys-data.bonsai (100%) create mode 100644 workflows/tutorials/basic-ephys-processing/select-convert-ephys-data.bonsai create mode 100644 workflows/tutorials/basic-ephys-processing/select-reorder-channels.bonsai rename workflows/tutorials/{spikes => basic-ephys-processing}/spike-detection.bonsai (100%) rename workflows/tutorials/{spikes => basic-ephys-processing}/spikes.bonsai (100%) diff --git a/articles/tutorials/spikes.md b/articles/tutorials/basic-ephys-processing.md similarity index 90% rename from articles/tutorials/spikes.md rename to articles/tutorials/basic-ephys-processing.md index 1ab550fd..0c472b80 100644 --- a/articles/tutorials/spikes.md +++ b/articles/tutorials/basic-ephys-processing.md @@ -1,7 +1,6 @@ --- -uid: spikes -title: Processing ephys data in Bonsai - +uid: basic-ephys-processing +title: Basic Ephys Data Processing in Bonsai --- @@ -17,7 +16,7 @@ This type of processing is important for real-time feedback while monitoring dat This tutorial will guide you through building the following workflow: ::: workflow -![/workflows/tutorials/spikes.bonsai workflow](../../workflows/tutorials/spikes/spikes.bonsai) +![/workflows/tutorials/basic-ephys-processing/spikes.bonsai workflow](../../workflows/tutorials/basic-ephys-processing/spikes.bonsai) ::: > [!NOTE] @@ -39,7 +38,7 @@ you're using the latest software. ## Configure the hardware ::: workflow -![/workflows/tutorials/spikes/configuration.bonsai workflow](../../workflows/tutorials/spikes/configuration.bonsai) +![/workflows/tutorials/basic-ephys-processing/configuration.bonsai workflow](../../workflows/tutorials/basic-ephys-processing/configuration.bonsai) ::: Construct a [top-level configuration chain](xref:initialize-onicontext): place the @@ -52,7 +51,7 @@ only device used in this tutorial, so you could disable other devices on the hea ## Stream ephys data into Bonsai ::: workflow -![/workflows/tutorials/spikes/ephys-data.bonsai workflow](../../workflows/tutorials/spikes/ephys-data.bonsai) +![/workflows/tutorials/basic-ephys-processing/ephys-data.bonsai workflow](../../workflows/tutorials/basic-ephys-processing/ephys-data.bonsai) ::: Place the relevant operator to stream electrophysiology data from your headstage and select the relevant output @@ -72,7 +71,7 @@ Visualize the raw data to confirm the ephys data operator is streaming data. ## Select and reorder channels ::: workflow -![/workflows/tutorials/spikes/select-convert-ephys-data.bonsai workflow](../../workflows/tutorials/spikes/select-convert-ephys-data.bonsai) +![/workflows/tutorials/basic-ephys-processing/select-convert-ephys-data.bonsai workflow](../../workflows/tutorials/basic-ephys-processing/select-convert-ephys-data.bonsai) ::: Connect a operator to the electrophysiology data stream and edit its "Channels" property. @@ -82,7 +81,7 @@ Reorder channels by writing the channel numbers in the order in which you want t ## Convert data to physical units ::: workflow -![/workflows/tutorials/spikes/select-convert-ephys-data.bonsai workflow](../../workflows/tutorials/spikes/select-convert-ephys-data.bonsai) +![/workflows/tutorials/basic-ephys-processing/select-convert-ephys-data.bonsai workflow](../../workflows/tutorials/basic-ephys-processing/select-convert-ephys-data.bonsai) ::: 1. Center the dynamic range of the ADC signal around zero @@ -117,7 +116,7 @@ worked as expected, i.e. that the signal is centered around zero and that the va ## Apply a frequency filter ::: workflow -![/workflows/tutorials/spikes/filter-ephys-data.bonsai workflow](../../workflows/tutorials/spikes/filter-ephys-data.bonsai) +![/workflows/tutorials/basic-ephys-processing/filter-ephys-data.bonsai workflow](../../workflows/tutorials/basic-ephys-processing/filter-ephys-data.bonsai) ::: Connect a `FrequencyFilter` operator to the second `ConvertScale` operator and set its properties. @@ -135,7 +134,7 @@ Connect a `FrequencyFilter` operator to the second `ConvertScale` operator and s ## Detect events ::: workflow -![/workflows/tutorials/spikes/spikes.bonsai workflow](../../workflows/tutorials/spikes/spikes.bonsai) +![/workflows/tutorials/basic-ephys-processing/spikes.bonsai workflow](../../workflows/tutorials/basic-ephys-processing/spikes.bonsai) ::: Based on the amplitude of the signal on the selected channel, set a fixed threshold for detecting spikes. diff --git a/articles/tutorials/toc.yml b/articles/tutorials/toc.yml index fc0663bb..a9dd3fc2 100644 --- a/articles/tutorials/toc.yml +++ b/articles/tutorials/toc.yml @@ -1,4 +1,4 @@ - href: index.md items: [ - href: spikes.md + href: basic-ephys-processing.md ] diff --git a/template/public/main.css b/template/public/main.css index bedc732c..9727119b 100644 --- a/template/public/main.css +++ b/template/public/main.css @@ -60,41 +60,35 @@ figcaption { padding: 0.25rem; } -h1 { +/* h1 { font-weight: 500; - /* originally font-size: calc(1.375rem + 1.5vw) */ font-size: calc(2.2rem + 0.8vw); } h2 { font-weight: 480; - /* originally font-size: calc(1.325rem + .9vw) */ font-size: calc(2.0rem + 0.6vw); } h3 { font-weight: 460; - /* originally font-size calc(1.3 + 0.6vw); */ font-size: calc(1.8rem + 0.4vw); } h4 { font-weight: 440; - /* originally font-size calc(1.275 + 0.3vw); */ font-size: calc(1.6rem + 0.2vw); } h5 { font-weight: 420; - /* originally font-size: 1.25rem */ font-size: 1.4rem; } h6 { font-weight: 400; - /* originally font-size: 1rem; */ font-size: 1.2rem; -} +} */ /* This is for formatting markdown files specifically */ diff --git a/workflows/tutorials/spikes/configuration.bonsai b/workflows/tutorials/basic-ephys-processing/configuration.bonsai similarity index 100% rename from workflows/tutorials/spikes/configuration.bonsai rename to workflows/tutorials/basic-ephys-processing/configuration.bonsai diff --git a/workflows/tutorials/spikes/select-convert-ephys-data.bonsai b/workflows/tutorials/basic-ephys-processing/convert-units.bonsai similarity index 100% rename from workflows/tutorials/spikes/select-convert-ephys-data.bonsai rename to workflows/tutorials/basic-ephys-processing/convert-units.bonsai diff --git a/workflows/tutorials/spikes/ephys-data.bonsai b/workflows/tutorials/basic-ephys-processing/ephys-data.bonsai similarity index 100% rename from workflows/tutorials/spikes/ephys-data.bonsai rename to workflows/tutorials/basic-ephys-processing/ephys-data.bonsai diff --git a/workflows/tutorials/spikes/filter-ephys-data.bonsai b/workflows/tutorials/basic-ephys-processing/filter-ephys-data.bonsai similarity index 100% rename from workflows/tutorials/spikes/filter-ephys-data.bonsai rename to workflows/tutorials/basic-ephys-processing/filter-ephys-data.bonsai diff --git a/workflows/tutorials/basic-ephys-processing/select-convert-ephys-data.bonsai b/workflows/tutorials/basic-ephys-processing/select-convert-ephys-data.bonsai new file mode 100644 index 00000000..98f8d0c0 --- /dev/null +++ b/workflows/tutorials/basic-ephys-processing/select-convert-ephys-data.bonsai @@ -0,0 +1,54 @@ + + + + + + + Headstage64/Rhd2164 + 30 + + + + Clock + + + AmplifierData + + + + + 0 + 1 + 2 + 3 + + + + + + F32 + 1 + -32768 + + + + + F32 + 0.195 + 0 + + + + + + + + + + + + \ No newline at end of file diff --git a/workflows/tutorials/basic-ephys-processing/select-reorder-channels.bonsai b/workflows/tutorials/basic-ephys-processing/select-reorder-channels.bonsai new file mode 100644 index 00000000..ab5b4241 --- /dev/null +++ b/workflows/tutorials/basic-ephys-processing/select-reorder-channels.bonsai @@ -0,0 +1,38 @@ + + + + + + + Headstage64/Rhd2164 + 30 + + + + Clock + + + AmplifierData + + + + + 0 + 1 + 2 + 3 + + + + + + + + + + + \ No newline at end of file diff --git a/workflows/tutorials/spikes/spike-detection.bonsai b/workflows/tutorials/basic-ephys-processing/spike-detection.bonsai similarity index 100% rename from workflows/tutorials/spikes/spike-detection.bonsai rename to workflows/tutorials/basic-ephys-processing/spike-detection.bonsai diff --git a/workflows/tutorials/spikes/spikes.bonsai b/workflows/tutorials/basic-ephys-processing/spikes.bonsai similarity index 100% rename from workflows/tutorials/spikes/spikes.bonsai rename to workflows/tutorials/basic-ephys-processing/spikes.bonsai From edd07236698f4874c18d6877f2cf82e34dc545ee Mon Sep 17 00:00:00 2001 From: ChucklesOnGitHub Date: Tue, 12 Nov 2024 17:30:22 +0000 Subject: [PATCH 15/19] style changes --- articles/tutorials/spikes.md | 64 +++++++++++++++++++----------------- 1 file changed, 33 insertions(+), 31 deletions(-) diff --git a/articles/tutorials/spikes.md b/articles/tutorials/spikes.md index 1ab550fd..f1a1c6e6 100644 --- a/articles/tutorials/spikes.md +++ b/articles/tutorials/spikes.md @@ -6,8 +6,7 @@ title: Processing ephys data in Bonsai -This tutorial shows how to use ONIX hardware and the OpenEphys.Onix1 Bonsai package to perform basic online signal -on electrophysiology data in Bonsai such as channel selection and reordering, frequency filtering and event detection (in this example, spike detection using a fixed threshold crossing). +This tutorial shows how to use ONIX hardware and the OpenEphys.Onix1 Bonsai package to perform basic online signal processing on electrophysiology data in Bonsai such as channel selection and reordering, frequency filtering and event detection (in this example, spike detection using a fixed threshold crossing). This type of processing is important for real-time feedback while monitoring data during acquisition. However, Bonsai's integrated visualizers and event detectors are limited, so we recommend using the tools available in the Open Ephys GUI for specialized ephys visualizations. The Open Ephys GUI is particularly suited for visualizing data from very dense arrays such as Neuropixels probes. @@ -33,7 +32,6 @@ Follow the [Getting Started](xref:getting-started) guide to set up and get famil - [Download the necessary Bonsai packages](xref:install-configure-bonsai#install-packages-in-bonsai) or [check for updates](xref:install-configure-bonsai#update-packages-in-bonsai). This tutorial assumes you're using the latest software. - - Read about [visualizing data](xref:visualize-data) since we recommend checking each step of the tutorial by visualizing the data produced but we don't cover it here. ## Configure the hardware @@ -42,10 +40,12 @@ you're using the latest software. ![/workflows/tutorials/spikes/configuration.bonsai workflow](../../workflows/tutorials/spikes/configuration.bonsai) ::: -Construct a [top-level configuration chain](xref:initialize-onicontext): place the +1. Construct a [top-level configuration chain](xref:initialize-onicontext): place the [configuration operators](xref:configure) that correspond to the hardware you intend to use between and . In this example, these are - and . Confirm that the device + and . + +1. Confirm that the device that streams electrophysiology data is enabled. The Rhd2164 device (an Intan amplifier) on the headstage64 is the only device used in this tutorial, so you could disable other devices on the headstage and on the breakout board to improve performance if you wanted to. @@ -55,15 +55,15 @@ only device used in this tutorial, so you could disable other devices on the hea ![/workflows/tutorials/spikes/ephys-data.bonsai workflow](../../workflows/tutorials/spikes/ephys-data.bonsai) ::: -Place the relevant operator to stream electrophysiology data from your headstage and select the relevant output -members. Because the device on headstage64 that streams electrophysiology data is the Rhd2164 Intan amplifier, we -placed the node onto the workflow. Select the relevant members from the data -frames that the data operator produces. In this example, the relevant members are "AmplifierData" and "Clock". To select those members, right-click the `Rhd2164` node, hover over the output option in the context menu, and select it from +1. Place the relevant operator to stream electrophysiology data from your headstage and select the relevant output members. Because the device on headstage64 that streams electrophysiology data is the Rhd2164 Intan amplifier, we +placed the node onto the workflow. + +1. Select the relevant members from the data frames that the data operator produces. In this example, the relevant members are "AmplifierData" and "Clock". To select those members, right-click the `Rhd2164` node, hover over the output option in the context menu, and select it from the list. - +1. Visualize the raw data to confirm that the ephys data operator is streaming data. -Visualize the raw data to confirm the ephys data operator is streaming data. + @@ -76,32 +76,32 @@ Visualize the raw data to confirm the ephys data operator is streaming data. ::: Connect a operator to the electrophysiology data stream and edit its "Channels" property. -Remember indexing in Bonsai starts at 0. Use commas to separate multiple channels and brackets for ranges. -Reorder channels by writing the channel numbers in the order in which you want to visualize the channels. -## Convert data to physical units +- Remember indexing in Bonsai starts at 0. +- Use commas to list multiple channels and brackets for ranges. +- Reorder channels by listing the channel numbers in the order in which you want to visualize the channels. + +## Convert ephys data to microvolts ::: workflow ![/workflows/tutorials/spikes/select-convert-ephys-data.bonsai workflow](../../workflows/tutorials/spikes/select-convert-ephys-data.bonsai) ::: -1. Center the dynamic range of the ADC signal around zero - - Connect a operator to the `SelectChannels` operator and set its properties: - - Edit its "Shift" property to subtract 2^bit depth - 1^ from the signal. Refer to the table at the bottom of - this tutorial to find the Shift necessary for each device.[^1] In this example, we "Shift" -32768 because the - Rhd2164 device outputs unsigned 16-bit data. - - Set the "Depth" property to F32 because this bit depth is required to correctly represent scaled data from all - devices. - -1. Scale the ADC signal to microvolts. - - Connect a second `ConvertScale` operator to the first `ConvertScale` operator and set its properties: - - Edit its "Scale" property to multiply the signal by a scalar in order to get microvolt values. This scalar is - determined by the gain of the amplifier and resolution the ADC contained in the amplifier device. Refer to the - table at the bottom of this tutorial to find the "Scale" necessary for each device.[^1] In this example, we - "Scale" by 0.195 because the Rhd2164 device on headstage64 has a step size of 0.195 μV/bit - - Keep the "Depth" property at F32. +### Center the signal around zero +Connect a operator to the `SelectChannels` operator and set its properties: +- Edit its "Shift" property to subtract 2^bit depth - 1^ from the signal. Refer to the table at the bottom of +this tutorial to find the Shift necessary for each device.[^1] In this example, we "Shift" -32768 because the +Rhd2164 device outputs unsigned 16-bit data. +- Set the "Depth" property to F32 because this bit depth is required to correctly represent scaled data from all +devices. + +### Scale the signal to microvolts +Connect a second `ConvertScale` operator to the first `ConvertScale` operator and set its properties: +- Edit its "Scale" property to multiply the signal by a scalar in order to get microvolt values. This scalar is +determined by the gain of the amplifier and resolution the ADC contained in the amplifier device. Refer to the +table at the bottom of this tutorial to find the "Scale" necessary for each device.[^1] In this example, we +"Scale" by 0.195 because the Rhd2164 device on headstage64 has a step size of 0.195 μV/bit +- Keep the "Depth" property at F32. Visualize the transformed data to confirm the output of the shifting and scaling operations worked as expected, i.e. that the signal is centered around zero and that the values make sense in microvolts. @@ -126,6 +126,8 @@ Connect a `FrequencyFilter` operator to the second `ConvertScale` operator and s - Set the "Cutoff1" and "Cutoff2" properties to an adequate value. In this example, we use 300 Hz as the lower cutoff frequency. +Visualize the filtered data. + > [!TIP] From b7244aedeb9ea6bfb891b78d9ad4f1c7e6234170 Mon Sep 17 00:00:00 2001 From: cjsha Date: Tue, 12 Nov 2024 12:48:36 -0500 Subject: [PATCH 16/19] Small edits for clarity --- articles/getting-started/index.md | 13 +++++++------ articles/hardware/index.md | 3 ++- articles/tutorials/index.md | 9 +++++---- 3 files changed, 14 insertions(+), 11 deletions(-) diff --git a/articles/getting-started/index.md b/articles/getting-started/index.md index 7565f4f3..e7b8dbf4 100644 --- a/articles/getting-started/index.md +++ b/articles/getting-started/index.md @@ -3,10 +3,11 @@ uid: getting-started title: Getting Started --- -Welcome to the user guide! The next few pages are dedicated to users who are unfamiliar with ONIX -and Bonsai, and will teach them what ONIX is, how to download and install Bonsai, open a new file, -place operators (and understand what an operator is), reorder a workflow, run a workflow, and -finally visualize data. +Welcome to the user guide! The next few pages are for users to learn how to use ONIX hardware using the OpenEphys.Onix1 +Bonsai package. Browse the articles in the table of contents to learn more. -For those who are already familiar with Bonsai and ONIX and are looking for a particular device or headstage -to learn about and how to utilize, visit the . \ No newline at end of file +This software documentation is intended to be used hand-in-hand with the +[Hardware documentation](https://open-ephys.github.io/onix-docs/index.html) to learn about and start using their ONIX hardware. + +For those who are looking for user guides and example workflows for using a particular device or headstage in Bonsai, visit +the . \ No newline at end of file diff --git a/articles/hardware/index.md b/articles/hardware/index.md index cbec537b..3689ae0c 100644 --- a/articles/hardware/index.md +++ b/articles/hardware/index.md @@ -3,4 +3,5 @@ uid: hardware title: Hardware Guides --- -Here you will find user guides for the ONIX breakout board, headstages, and other compatible hardware that are supported by the library. +Here you will find user guides and example workflows for the ONIX breakout board, headstages, and other compatible +hardware that are supported by the library. diff --git a/articles/tutorials/index.md b/articles/tutorials/index.md index 73f530f4..e8376f73 100644 --- a/articles/tutorials/index.md +++ b/articles/tutorials/index.md @@ -3,7 +3,8 @@ uid: tutorials title: Tutorials --- -> [!Note] -> This section will contains tutorials that demonstrate how to make the most of -> the Bonsai library by combining ONIX with third party tools, tuning closed loop -> response times, etc. \ No newline at end of file +This section contains tutorials that demonstrate how to make the most of +the Bonsai library by combining ONIX with third party software tools + + \ No newline at end of file From d701f7e56dc8ea9882d5bca03d55a00a191faaa5 Mon Sep 17 00:00:00 2001 From: cjsha Date: Tue, 12 Nov 2024 13:53:46 -0500 Subject: [PATCH 17/19] Edits for clarity --- articles/getting-started/visualize-data.md | 33 ++++++++++++++-------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/articles/getting-started/visualize-data.md b/articles/getting-started/visualize-data.md index f6ff4174..1e27ac63 100644 --- a/articles/getting-started/visualize-data.md +++ b/articles/getting-started/visualize-data.md @@ -3,17 +3,17 @@ uid: visualize-data title: Visualize Data --- -Bonsai has ["type visualizers"](https://bonsai-rx.org/docs/articles/editor.html?#type-visualizers) that open as individual windows when double-clicking a data node to display data produced by an operator while the workflow is running. +Bonsai has ["type visualizers"](https://bonsai-rx.org/docs/articles/editor.html?#type-visualizers) that display data +produced by an operator. They are opened by double-clicking the corresponding node while the workflow is running. -Not all nodes have type visualizers, because the operator data output type has to be compatible with Bonsai's default type visualizers. Data frames from the OpenEphys.Onix1 library, for example, aren't compatible with Bonsai's default type visualizers, but its data members are. After performing member selection on operators from the OpenEphys.Onix1 library, you can use the default type visualizers. - -Other visualizers for use with other libraries are available as standalone operators and can be found in the `Bonsai.Design.Visualizers` package, which can be installed in the Bonsai package manager. These visualizer operators must be placed in the workflow and linked to a data operator. The visualizer window can be opened by double-clicking the visualizer node instead of the preceding data node. +Read below for more details about how to visualize data. ## Selecting operator data members for visualization -If the node whose data you want to visualize requires member selection, such as ONIX [data I/O operators](xref:dataio): +Some operators, such as [ONIX data I/O operators](xref:dataio), require selecting members from their output to visualize +their data: 1. Right-click the node that corresponds to the data I/O operator you'd like to visualize. - 1. Hover over the "Output" option that appears in the context menu. + 1. Hover the cursor over the "Output" option that appears in the context menu. 1. Click the member you would like to visualize from the list of members. This populates the workflow with a operator that selects the single @@ -23,11 +23,16 @@ member from the data frame produced by the data I/O operator. +> [!NOTE] +> Member selection is required when an operator's output type doesn't have type visualizers that allow users to inspect +> the data in a meaningful capacity. This is true for [ONIX data I/O operators](xref:dataio) which typically produce +> [data frames](xref:data-elements). + ## Selecting visualizers -Select the visualizer you would like to use for the data you would like visualize: +Select the visualizer you would like to use for visualizing data: 1. Right-click the `MemberSelector` node labelled with the member you would like to visualize. - 1. Hover over the "Select Visualizer" option in the context menu. + 1. Hover the cursor over the "Select Visualizer" option in the context menu. 1. Click the visualizer you would like to use from the list of visualizers.