| title | description | date | author | tags |
|---|---|---|---|---|
IndexPyKX objects |
How to index PyKX object |
July 2024 |
KX Systems, Inc., |
PyKX, q, PyKX objects, index |
This page provides details on how indexing works within PyKX.
Indexing in q works differently than you may be used to, and that behavior largely carries over into PyKX for indexing K objects.
!!! info "Resources"
For more information about how indexing in q works (and by extension, how indexing K objects in PyKX work), refer to the following sections of the q tutorial book [Q For Mortals](https://code.kx.com/q4m3/):
- [Indexing](https://code.kx.com/q4m3/3_Lists/#34-indexing)
- [Iterated indexing and indexing at depth](https://code.kx.com/q4m3/3_Lists/#38-iterated-indexing-and-indexing-at-depth)
- [Indexing with lists](https://code.kx.com/q4m3/3_Lists/#39-indexing-with-lists)
- [Elided indices](https://code.kx.com/q4m3/3_Lists/#310-elided-indices)
Indexes used on K objects in PyKX are converted to equivalent K objects in q using the toq module, just like any other Python to q conversion. To guarantee that the index used against a K object is what you intend it to be, you may perform the conversion of the index yourself before applying it. When K objects are used as the index for another K object, the index object is applied to the [#!python pykx.Collection][pykx.Collection] object as it would be in q, for example as described in Q For Mortals.
Examples of applying indexing to various q objects include:
- a. Basic vectors indexing: Single element indexing and Slicing
- b. Assigning and adding values to vectors/lists
- c. Indexing non-vector objects
Indexing in PyKX spans elements #!python 0 to element #!python N-1 where #!python N is the length of the object being indexed.
Single element indexing works like any other standard Python sequence. Similar to Numpy, PyKX supports negative indices to allow retrieval of indexes at the end of an array. For example:
>>> x = kx.q.til(10)
>>> x[2]
pykx.LongAtom(pykx.q('2'))
>>> x[-2]
pykx.LongAtom(pykx.q('8'))
>>> y = kx.CharVector('abcdefg')
>>> y[0]
pykx.CharAtom(pykx.q('"a"'))
>>> y[-2]
pykx.CharAtom(pykx.q('"f"')) Similar to Numpy, indexing an array out of bounds results in an #!python IndexError being raised.
>>> x = kx.q.til(5)
>>> x[6]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 1165, in __getitem__
return q('@', self, _idx_to_k(key, _wrappers.k_n(self)))
File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 212, in _idx_to_k
raise IndexError('index out of range')
IndexError: index out of rangeN-dimensional list vectors can also be manipulated using single element indexing as follows:
>>> x = kx.random.random([4, 4], 1.0)
>>> x
pykx.List(pykx.q('
0.5294808 0.6916099 0.2296615 0.6919531
0.4707883 0.6346716 0.9672398 0.2306385
0.949975 0.439081 0.5759051 0.5919004
0.8481567 0.389056 0.391543 0.08123546
'))
>>> x[0][3]
pykx.FloatAtom(pykx.q('0.6919531'))Slicing vectors in PyKX is simpler than the functionality provided by Numpy. You can index vectors of N dimensions by using #!python obj[start:stop:step] semantics. This slice syntax operates where #!python start is the starting index, #!python stop is the stopping index and #!python step is the number of steps between the elements where #!python step is non zero:
>>> x = kx.q.til(10)
>>> x[2:4]
pykx.LongVector(pykx.q('2 3'))
>>> x[5:]
pykx.LongVector(pykx.q('5 6 7 8 9'))
>>> x[:8:2]
pykx.LongVector(pykx.q('0 2 4 6'))
>>> x = kx.random.random([4, 4], 1.0)
>>> x[:2]
pykx.List(pykx.q('
0.1477547 0.274227 0.5635053 0.883823
0.2439194 0.6718125 0.8639591 0.8439807
'))
>>> y = kx.CharVector('abcdefg')
>>> y[2:4]
pykx.CharVector(pykx.q('"cd"'))
>>> y[3:]
pykx.CharVector(pykx.q('"defg"'))
>>> y[:6:2]
pykx.CharVector(pykx.q('"ace"'))Negative slicing works in a similar way. You can use it for #!python list, #!python vector and #!python table objects, too.
>>> list = kx.q('("a"; 2; 3.3; `four)')
>>> list[-3:]
pykx.List(pykx.q('
2
3.3
`four
'))
>>> vector = kx.q.til(5)
>>> vector[:-1]
pykx.LongVector(pykx.q('0 1 2 3'))
>>> table = kx.Table(data = {
... 'a': [1, 2, 3],
... 'b': [4, 5, 6],
... 'c': [7, 8, 9],
... })
>>> table[-2:]
pykx.Table(pykx.q('
a b c
-----
2 5 8
3 6 9
'))Vector assignment in PyKX operates similarly to that provided by Numpy and operations supported on basic Python lists. As with the previous sections this functionality supports both individual element assignment and slice assignment as follows:
>>> vec = kx.q.til(10)
>>> vec
pykx.LongVector(pykx.q('0 1 2 3 4 5 6 7 8 9'))
>>> vec[1] = 15
>>> vec
pykx.LongVector(pykx.q('0 15 2 3 4 5 6 7 8 9'))
>>> vec[-1] = 10
>>> vec
pykx.LongVector(pykx.q('0 15 2 3 4 5 6 7 8 10'))
>>> vec[:5] = 0
>>> vec
pykx.LongVector(pykx.q('0 0 0 0 0 5 6 7 8 10'))??? Note "N-Dimensional vector element assignment not supported"
```python
>>> x = kx.random.random([4, 4], 1.0)
>>> x
pykx.List(pykx.q('
0.3927524 0.5170911 0.5159796 0.4066642
0.1780839 0.3017723 0.785033 0.5347096
0.7111716 0.411597 0.4931835 0.5785203
0.08388858 0.1959907 0.375638 0.6137452
'))
>>> x[0][2] = 3.0
>>> x
pykx.List(pykx.q('
0.3927524 0.5170911 0.5159796 0.4066642
0.1780839 0.3017723 0.785033 0.5347096
0.7111716 0.411597 0.4931835 0.5785203
0.08388858 0.1959907 0.375638 0.6137452
'))
```
In addition to positional assignment, you can use the #!python append and #!python extend methods for #!python pykx.*Vector and #!python pykx.List objects. When adding objects to a list, use #!python append for single item assignments. In contrast, use #!python extend to add multiple elements to a #!python list or #!python vector object. The following tabbed section demonstrates the use of #!python append and #!python extend operations, including examples of failing cases.
=== "pykx.*Vector"
```python
>>> import pykx as kx
>>> qvec = kx.random.random(3, 1.0, seed = 42)
>>> qvec
pykx.FloatVector(pykx.q('0.7742128 0.7049724 0.5212126'))
>>> qvec.append(1.1)
>>> qvec
pykx.FloatVector(pykx.q('0.7742128 0.7049724 0.5212126 1.1'))
>>>
>>> qvec.append([1.2, 1.3, 1.4])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 1262, in append
raise QError(f'Appending data of type: {type(data)} '
pykx.exceptions.QError: Appending data of type: <class 'pykx.wrappers.FloatVector'> to vector of type: <class 'pykx.wrappers.FloatVector'> not supported
>>>
>>> qvec.extend([1.2, 1.3, 1.4])
pykx.FloatVector(pykx.q('0.7742128 0.7049724 0.5212126 1.1 1.2 1.3 1.4'))
>>>
>>> qvec.extend([1, 2, 3])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/wrappers.py", line 1271, in extend
raise QError(f'Extending data of type: {type(data)} '
pykx.exceptions.QError: Extending data of type: <class 'pykx.wrappers.LongVector'> to vector of type: <class 'pykx.wrappers.FloatVector'> not supported
```
=== "pykx.List"
```python
>>> qlist = kx.toq([1, 2, 1.3])
>>> qlist
pykx.List(pykx.q('
1
2
1.3
'))
>>> qlist.append({'x': 1})
>>> qlist
pykx.List(pykx.q('
1
2
1.3
(,`x)!,1
'))
>>> qlist.extend([1, 2])
>>> qlist
pykx.List(pykx.q('
1
2
1.3
(,`x)!,1
1
2
'))
```
In addition to being able to index and slice PyKX #!python vector and #!python list objects, it's also possible to apply index and slicing semantics on PyKX Table objects. Application of slice/index semantics on tabular objects returns table-like objects:
>>> import pandas as pd
>>> from random import random
>>> from uuid import uuid4
>>> df = pd.DataFrame.from_dict({
'x': [random() for _ in range(10)],
'x1': [random() for _ in range(10)],
'x2': [uuid4() for _ in range(10)]
})
>>> tab = kx.toq(df)
>>> tab
pykx.Table(pykx.q('
x x1 x2
-----------------------------------------------------------
0.1872634 0.4176994 c9555cdf-57db-28a8-bf6c-67f6ee711a5f
0.8416288 0.01920741 3ecca92c-aae6-f796-38c9-80f4da70a89d
0.7250709 0.8761714 6417d4b3-3fc6-e35a-1c34-8c5c3327b1e8
0.481804 0.7575856 4040cd34-c49e-587b-e546-e1342bf1dd85
0.9351307 0.6030223 e8327955-bd9a-246a-0b17-fbf8f05fd28a
0.7093398 0.1811364 54a1959c-997c-6c57-1ff2-a0f3e845f01d
0.9452199 0.2329662 008bded0-3383-f19b-1d18-abfb199b1ac1
0.7092423 0.250046 6f54a161-49e7-f707-0054-626b867fb02f
0.002184472 0.0737272 f294c3cb-a6da-e15d-c8e0-3a848d2abf10
0.06670537 0.3186642 cd17ee98-c089-10a3-8992-d437a566f081
'))
>>> tab[3]
x x1 x2
-----------------------------------------------------------
0.481804 0.7575856 4040cd34-c49e-587b-e546-e1342bf1dd85
'))
>>> tab[:5]
pykx.Table(pykx.q('
x x1 x2
---------------------------------------------------------
0.1872634 0.4176994 c9555cdf-57db-28a8-bf6c-67f6ee711a5f
0.8416288 0.01920741 3ecca92c-aae6-f796-38c9-80f4da70a89d
0.7250709 0.8761714 6417d4b3-3fc6-e35a-1c34-8c5c3327b1e8
0.481804 0.7575856 4040cd34-c49e-587b-e546-e1342bf1dd85
0.9351307 0.6030223 e8327955-bd9a-246a-0b17-fbf8f05fd28a
'))
>>> tab[0:8:2]
pykx.Table(pykx.q('
x x1 x2
--------------------------------------------------------
0.1872634 0.4176994 c9555cdf-57db-28a8-bf6c-67f6ee711a5f
0.7250709 0.8761714 6417d4b3-3fc6-e35a-1c34-8c5c3327b1e8
0.9351307 0.6030223 e8327955-bd9a-246a-0b17-fbf8f05fd28a
0.9452199 0.2329662 008bded0-3383-f19b-1d18-abfb199b1ac1
'))