Refactor channel search functionality in tdm_loader (#28)

* Refactor channel search functionality in `tdm_loader`

The previous implementation of the channel search function in the `tdm_loader.py` was refactored. Search now matches channels directly without set generation and updates retrieval with more efficient lookups. The modification updates the testing to reflect the changes in the returned search results as well.

Before the refactor searching throe 22000 channels took 6.5min and after it took 0.08s

Also the test has changed to be similar when searching for "", all channels should be returned.

* Update tdm_loader/tdm_loader.py

Co-authored-by: Florian Dobener <[email protected]>

* Update tdm_loader/tdm_loader.py

Co-authored-by: Florian Dobener <[email protected]>

* Update tdm_loader/tdm_loader.py

Co-authored-by: Florian Dobener <[email protected]>

* Update tdm_loader/tdm_loader.py

Co-authored-by: Florian Dobener <[email protected]>

* Update tdm_loader/tdm_loader.py

Co-authored-by: Florian Dobener <[email protected]>

* Update tdm_loader/tdm_loader.py

Co-authored-by: Florian Dobener <[email protected]>

* Implement caching and optimize channel search in `tdm_loader`

A cache decorator is introduced for the `get_channels` function to speed up repeated lookups, replacing the previous, dictionary lookup.

* Rename method get_channels to _get_channels in tdm_loader

The public `get_channels` function is renamed to the private method `_get_channels` in 'tdm_loader.py'. This is due to internal use only, to provide more clarity and prevent unintended external access.

* Correct typo in method name in tdm_loader

The method name "get_channels" has been corrected to "_get_channels" in the 'tdm_loader.py' file. The change was necessitated by the requirement for the method to be private, hence the underscore, as it's only meant for internal use, preventing unintended external access.

---------

Co-authored-by: Espen Enes <[email protected]>
Co-authored-by: Florian Dobener <[email protected]>

Author: 3 people

tdm_loader/tdm_loader.py

@@ -26,6 +26,7 @@
 import os
 import zipfile
 import re
+from functools import cache
 
 from xml.etree import ElementTree
 import warnings
@@ -168,6 +169,11 @@ def _get_usi_from_txt(txt):
             return []
         return re.findall(r'id\("(.+?)"\)', txt)
 
+    @cache
+    def _get_channels(self, group_id):
+        group = self._xml_chgs[group_id]
+        return {v: i for i, v in enumerate(re.findall(r'id\("(.+?)"\)', group.find("channels").text))}
+
     def channel_group_search(self, search_term):
         """Returns a list of channel group names that contain ``search term``.
         Results are independent of case and spaces in the channel name.
@@ -228,31 +234,22 @@ def channel_search(self, search_term):
         """
         search_term = str(search_term).upper().replace(" ", "")
 
-        ind_chg_ch = []
-        for j in range(len(self._xml_chgs)):
-            chs = self._channels_xml(j)
+        matched_channels = []
+        channel_group_ids = {v: i for i, v in enumerate(x.get("id") for x in self._xml_chgs)}
 
-            if search_term == "":
-                found_terms = [
-                    ch.findtext("name") for ch in chs if ch.findtext("name") is None
-                ]
-            else:
-                found_terms = [
-                    ch.findtext("name")
-                    for ch in chs
-                    if ch.findtext("name") is not None
-                    and ch.findtext("name")
-                    .upper()
-                    .replace(" ", "")
-                    .find(str(search_term))
-                    >= 0
-                ]
-
-            for name in found_terms:
-                i = [ch.findtext("name") for ch in chs].index(name)
-                ind_chg_ch.append((name, j, i))
-
-        return ind_chg_ch
+        for channel in self._root.findall(".//tdm_channel"):
+            channel_name = channel.find("name").text
+            if channel_name:
+                group_uri = re.findall(r'id\("(.+?)"\)', channel.find("group").text)
+                group_id = channel_group_ids.get(group_uri[0])
+                channels = self._get_channels(group_id)
+
+                channel_id = channels.get(channel.get("id"))
+
+                if channel_name.upper().replace(" ", "").find(search_term) >= 0:
+                    matched_channels.append((channel_name, group_id, channel_id))
+
+        return matched_channels
 
     def channel(self, channel_group, channel, occurrence=0, ch_occurrence=0):
         """Returns a data channel by its channel group and channel index.

tdm_loader/tests/test_non_zip_tdm.py

@@ -204,7 +204,9 @@ def test_channel_search(tdm_file):
         ("Float as Float", 0, 1),
     ]
 
-    assert tdm_file.channel_search("") == []
+    assert tdm_file.channel_search("") == [('Float_4_Integers', 0, 0),
+                                           ('Float as Float', 0, 1),
+                                           ('Integer32_with_max_min', 0, 2)]
 
 
 # pylint: disable=redefined-outer-name