Structures

THIS DOCUMENT DESCRIBES AN UPCOMING FEATURE OF CTYPES.SH VERSION 1.1

Bash doesn't provide any way to interact with native data structures, so ctypes.sh will translate native structures into bash format and back again. This process is called packing and unpacking.

In many cases, ctypes.sh can automatically import structures from the libraries you want to use. This makes working with structures in bash simple and easy.

Automatic Structure Definition

To define a structure in bash, use the struct command. The parameters are the name of a structure defined a library you have loaded via dlopen, followed by the name of the variable to contain the structure.

$ struct stat s

This example creates a bash version of the standard stat structure in the variable s. To access the members of the structure, specify the member name in brackets, like ${s[st_size]}. You can also access the members of nested structures, just use . between names. For example, the stat structure stores the last access time in a nested structure called st_atim, access the seconds part like this ${s[st_atim.tv_sec]}.

If automatic structure definition doesn't work, don't worry, there are troubleshooting steps below.

Let's look at a complete example.

$ dlcall -n statbuf -r pointer malloc $(sizeof stat)
pointer:0x734608
$ struct stat passwd
$ dlcall -r int __xstat 0 "/etc/passwd" $statbuf
int:0
$ unpack $statbuf passwd
$ echo ${passwd[st_mtim.tv_sec]}
long:1446019680
$ date --date=@1446019680
Wed Oct 28 01:08:00 PDT 2015
$ stat -c %y /etc/passwd
2015-10-28 01:08:00.843728968 -0700

1. In this example we use the sizeof command to allocate memory with malloc.
2. Call stat() on /etc/passwd (note that on Linux, __xstat is used instead of stat)
3. Use unpack to translate the result into a bash structure.

From here we can access members and verify that we got the data we expected.

Unions

Most structures can be imported automatically, but structures containing unions can be ambiguous. That is, they could be imported in many different ways. By default, the first member of a union is always used.

If that is not what you want, you need to override the default parsing with the -u option.

Let's look at an example, consider a structure like this:

struct whatever {
 union {
  int a;
  float b;
 } blah;
};

If you want ${whatever[blah.b]} rather than ${whatever[blah.a]}, you need to override the parsing like this:

$ struct -u blah:b whatever whatever

Separate more unions with commas, for example struct -u blah:b,foo:x,bar.baz:y whatever foo

Anonymous Unions

It is common for programmers to use anonymous unions in C. This is only legal when the result is unambiguous, so you can just omit the union name. For example, the following structure members could be addressed like ${whatever[.a]} and ${whatever[.b]}.

struct {
 union { int a };
 union { long b };
}

Troubleshooting Automatic Structures

Automatic structure definition works by parsing the DWARF debugging data in libraries. ctypes.sh includes data for many standard UNIX types already, but if you are using another library you may need to install the debugging data.

• On Fedora, RedHat or CentOS, try debuginfo-install <library>
• On Debian or Ubuntu, try apt-get install <library>-dbg
• On FreeBSD, enable WITH_DEBUG_FILES in src.conf and recompile
• If this is your own library, don't use strip or gcc -s

Many standard structures are part of glibc, try debuginfo-install glibc or apt-get install libc-dbg.

If none of these options are possible, you can either define the structure manually, or build a small shared object.

Creating a shared object

If you do not have debugging data for your library, but you do have the include files for it, you can build a small shared object and ctypes.sh can import the structures from it.

Create a .c file that just creates the structure you are using, the entire file is shown below.

$ cat example.c
#include <yourlibrary.h>

struct yourstruct test;

Compile that c file with debugging data.

$ cc -g -shared -o example.so example.c

Now load that file into bash so that ctypes.sh knows to search it.

$ dlopen ./example.so
0x234180

Your structure should now be available with the struct command.

Caveats

Some features like bitfields and complex multi-dimensional arrays can be difficult to parse, if the result is incorrect or ctypes.sh cannot parse your structure correctly, please file a bug.

Technical Notes

The struct command creates an unusual bash data structure.

You're probably familiar with bash's indexed arrays (declare -a) and associative arrays (declare -A). Internally, bash stores associative arrays as hash tables, which means the order of elements is discarded. You can test this yourself:

$ declare -A hello
$ hello[foo]=1 hello[bar]=2 hello[baz]=3 hello[quz]=4 
$ echo ${hello[@]}
2 4 3 1

However, bash's internal API allows you to create associative arrays with an arbitrary bucket size. ctypes.sh uses this feature to create associative arrays that maintain their order, by setting the bucket size to 1.

This means the associative arrays created by struct are technically hash tables, but work like linked lists. Therefore, they will maintain the order of elements.

Because of this, arrays created by the struct command may act differently than the arrays created with declare -A. ctypes.sh does this so that the more natural associative array syntax can be used with structures.

Manual Structure Definition

If you cannot use automatic structure definition, you can define your structure manually in an indexed array.

    $ data=(int int long pointer)
    $ unpack pointer:0x9e2d90 data
    $ echo ${data[@]}
    int:1460145432 int:32586 long:10350560 pointer:(nil)

Packing and Unpacking

ctypes.sh provides two commands for interacting with structs, pack and unpack.

• pack converts from an array of prefixed types to native format.
• unpack converts a pointer to an array of prefixed types.

In its simplest form, you specify the types and a pointer to fetch them from

    $ data=(int int long pointer)
    $ unpack pointer:0x9e2d90 data
    $ echo ${data[@]}
    int:1460145432 int:32586 long:10350560 pointer:(nil)