Getting started in driver development

[anto31ad]

A little backstory

I've grown tired of Windows over the last years and some months ago I wanted to make the permanent switch to the FOSS and libre world, starting from linux; obviously that led to some minor inconveniences when it came to adapt some workflows to linux, like gaming.

So i had a T300 lying around and wanted to get it to work on my linux system, and this was the first repo I was redirected to;

Now, I've been going through a phase of my life where I wasn't really sure what kind of skills to build to find a future occupation that I would like enough.

Only recently I decided that among other options, programming skills were the ones I would focus on for a while, because all in all I've enjoyed the coding I did so far and it's also a relevant skill at this time in history.

But saying I wanted to do programming is still to general, since there's multiple subfields to consider;

So far,

• Got my hands wet in a couple web development projects but I quit both because, among all reasons, I wasn't enjoying it that much.
• I was able to code a little car game with Unity as a uni project and I enjoyed it a lot (heres a demo video), especially coding the ray wheel model from scratch.

In both cases though, I didn't like the fact I was using tools that, through heavy abstraction, hide what happens under the hood, and lately I've slowly been craving to learn about how computers work and write programs that run at a low level;

Coming across repos like this one started to kinda fascinate me, when in the past I would have just run away at the sight of something difficult to understand. Not that that went completely away, but I'm fighting it, because it's rewarding afterwards.

A challenge to take on

I have a T-LCM set of pedals that seem to work fine when plugged through RJ cable to the wheelbase but are not recognized by programs like InputRemapper (so also games, according to other owner) when connected via USB.

I suppose there's no kernel module that handles this device. So I thought, why shouldn't I try to solve this problem?

Question time

So as mentioned in issue #101 , I wanted to ask @Kimplul a couple of questions that might be of interest of people wanting to get started building low level programming skills;

1. First of all, I was curious how this project started and what skills you could count on in the beginning.
2. Secondly, how can someone get started in developing a driver (for linux in this case)?

Notes on 2)

I found various resources online, one of which the free book Linux Device Drivers, not sure if that was a reference for your development as well 😄 .

[Kimplul]

Sounds a bit like where I was four or five years ago. I'll answer the questions
and waffle on about related things as a response to the post as a whole, hope
this isn't too incomprehensible.

In both cases though, I didn't like the fact I was using tools that, through
heavy abstraction, hide what happens under the hood, and lately I've slowly
been craving to learn about how computers work and write programs that run at
a low level;

Yep, I also like seeing how the sausage is made. I would guess that it was this
project (and that it didn't immediately explode in my face) that got me more
interested in lower levels of programming and computer architecture details, to
the extent that I later started writing my own OS
(https://github.com/Kimplul/kmi) and CPU architecture
(https://github.com/Kimplul/tri). Both are of course just hobby projects, no
intention of getting them into any real use anywhere, just something I do when
I'm bored, but there are some aspects you could trace to this project.

You seem to be interested in possible career options, so it's probably worth
mentioning that I'm majoring in embedded systems at university and have for a
few summers now helped develop a simulator toolset for application specific
instruction sets for custom processors. Maybe not directly because of this
project, but again, probably some traceable aspects.

Coming across repos like this one started to kinda fascinate me, when in the
past I would have just run away at the sight of something difficult to
understand. Not that that went completely away, but I'm fighting it, because
it's rewarding afterwards.

Oh yeah, massively rewarding. You've probably already heard something along
these lines (and I expand upon it below) but generally the best way to get
started with a big project is to try and focus on some small aspect and just
start working on it, slowly expanding out to other areas and eventually
the whole thing is conquered. Can generally be applied to any task, but I find
that it's particularly fitting with computer science/engineering.

First of all, I was curious how this project started and what skills you could
count on in the beginning.

The main reason I started this project was that I had fairly recently switched
to using Linux as a daily driver only, and was frustrated that I couldn't use my
T300 to play DiRT Rally. This happened at kind of a lucky time, as I had just
finished my matriculation exams and had a little under a half a year before my
university studies would start, so I had the motivation and time to take on what
I considered a 'big' project. Not necessarily 'big' as in in lines of code, but
just something I hadn't done before, didn't know a lot about, wasn't sure if it
was going to succeed or not.

I had been casually programming for maybe a couple years before that point,
initially some web dev kind of 'introduction to programming' on some website but
I got bored of that fairly quickly and moved on to C++ as I thought I wanted to
make games. Got an OpenGL paint-by-numbers kind of book, but that didn't really
take either. So, in other words, very little development experience. Luckily I
was at least somewhat aware of how C works due to having used a bit of C++. I
believe I knew what a kernel was, at least along the lines of 'it has the right
to do anything', but didn't really know anything about implementation details.

Secondly, how can someone get started in developing a driver (for linux in
this case)?

I guess I'll write what steps I more-or-less went through, though I want to
stress that this is my only driver (for Linux, at least) and I'm no greybeard
with a hundred worn out keyboards in my closet.
Generally most drivers would also be integrated into the
kernel, so I don't really have a lot of experience with that side of Linux
driver development.

Anycase, I started out with just trying to figure out how some tiny bit of wheel
functionality worked. I spent a lot of time capturing different USB packets with
Wireshark and tried to find patterns in the data, and wrote some very crude
module files that just tried to do one thing. I have a rough outline of the
packet capture process in the wiki.

For the most part actually writing the modules was not a huge issue, luckily
Linux is open source so I glanced through different drivers for similar devices
and more or less copied parts that I thought would fit. This was good enough for
producing some prototypes and getting a feel for the device, and in hindsight
was probably good call as I had a huge number of wrong assumptions about how the
wheel worked, and did a lot rewriting. At this point the driver wasn't
publicly available and I wasn't too worried about ugly code or
maintainability, just interested in getting it to work for me. Nor did I really
have a particularly huge interest in writing 'correct' code, just clobber
something together that works. This of course led to numerous kernel panics, so
a virtual machine was and is pretty vital.

At some point I thought I had a good enough overview of how the wheel functions
and what the driver would have to look like, so I started doing
a rewrite into something that I felt better about putting out in public. I felt
I should probably actually understand what the code I'm using does, and spent a
while reading documentation and trying to gain an understanding of what I was
doing. For the most part, the Linux documentation was very helpful, and any gaps
I had were generally filled in by reading through the Linux kernel source code.
The stuff I couldn't find documentation on and couldn't understand the code for,
I again just wrote some prototypes and tried to figure out what worked and what
didn't. Luckily there weren't particularly many situations like that.

At some point this thing started gaining a small amount of users and I got some
suggestions for how to improve things, and I (again) did a rewrite when I
started adding support for more wheels than just the T300.

I realize this may sound like climbing up a tree ass first, and I'm sure for
some it would be more natural to try and first get an understanding of what
you're working on and then try to write stuff. I still find that writing code is
the best way to process a given problem, focus on some small part and try to
write things in such a way that it's easy to give up some approach and rewrite
it to try another angle. Probably not what you want to do when lives are at
stake, but good enough for hobby projects.

For what it's worth, a lot of this is applicable to more than just drivers, but if
nothing else at least take this as a testament that some of the commonly heard
platitudes seem to generally work.
Don't be afraid to try again, take small steps where
possible, doing something is better than doing nothing, stuff like that.

I found various resources online, one of which the free book Linux Device
Drivers, not sure if that was a reference for your development as well 😄 .

Good book, I think I remember reading the chapter on USB drivers. Turned out to not
really be that applicable to this project, but it was well written.

I unfortunately don't really have a good list of reading material or anything
along those lines, I don't intend to gatekeep or be one of those 'do your own
research' kinds but I've generally just found the Linux documentation and source
code to be good enough and no guide or similar has really stood out to me, as far as the actual programming is concerned. Seeing that you're probably going to be dealing with a USB device, I would recommend getting a rough feel for the USB API in the kernel, and you might want to be aware that there's a separate Human Interface Device (HID) specification built on top of USB that the pedals may or may not use.

I suppose there's no kernel module that handles this device. So I thought, why
shouldn't I try to solve this problem?

Excellent attitude.

If you have any follow-up questions, feel free to ask. I went a bit long with these open-ended questions but I promise I'll try to be more concise in the future.

[anto31ad]

Sorry for the late reply, these weeks were and will be quite busy!

You seem to be interested in possible career options, so it's probably worth
mentioning that I'm majoring in embedded systems at university and have for a
few summers now helped develop a simulator toolset for application specific
instruction sets for custom processors.

I suppose this is a Master's degree, right?

You've probably already heard something along
these lines (and I expand upon it below) but generally the best way to get
started with a big project is to try and focus on some small aspect and just
start working on it, slowly expanding out to other areas and eventually
the whole thing is conquered

Oh yes, probably heard something like this many times so far in different forms, but only recently it really started to click, as if I now have the tools to really get what it means;
still didn't really put it into practice though, as I have to fight this very bad habit of thinking too big at the wrong times, ahah!

just something I hadn't done before, didn't know a lot about, wasn't sure if it
was going to succeed or not.

When I read this I just thought that it's cool that it all starts like this for almost everyone;
The biggest takeaway I get is that you just gotta persevere until the ball starts rolling, but I have had trouble with this many times. I think this follows from the previous point: I expect to do too much at the same time, and when I eventually "fail" I put myself into the position to quit early.
I see this happening in real time when I am coding, where I work in too many areas at once and end up with no concrete, testable code at the end of the session.

I spent a lot of time capturing different USB packets with
Wireshark and tried to find patterns in the data

This is kinda what still scares me a bit, at this stage. The investigation tasks. Because at the beginning you're not sure where you're going and if it's worth it. But I managed to perform this kind of tasks (generally speaking) and I was very happy every time I succeeded. Yet I still have trouble to start such tasks;

That aside, I first heard about Wireshark from a CTF team in my university when they held a little seminar, I thought it was a tool for watching data transfers in a network, but I guess that also applies for peripherals;
And I suppose what you did is reverse engineering the driver by looking at the type of data that the wheel transfers when plugged.
You had to use Windows to perform these tasks, right? Because I'm led to think that you need to study the device when it functioning, to reverse engineer it.

I unfortunately don't really have a good list of reading material or anything
along those lines, I don't intend to gatekeep or be one of those 'do your own
research' kinds but I've generally just found the Linux documentation and source
code to be good enough and no guide or similar has really stood out to me, as far as the actual programming is concerned.

I wouldn't think that of you! At the end, people learn the most by doing, and references are meant to support this trial and error process.

[Kimplul]

I suppose this is a Master's degree, right?

Yeah, though I started with the group I work at just before I got my Bachelor's.

That aside, I first heard about Wireshark from a CTF team in my university when they held a little seminar, I thought it was a tool for watching data transfers in a network, but I guess that also applies for peripherals;

Yeah, there's a guide for doing USB captures with Wireshark over at https://wiki.wireshark.org/CaptureSetup/USB. There are also actual hardware capture cards for doing the same thing, though the cost seems to dramatically go up with USB transfer speeds. Don't really know what if any drawbacks a software capture has over hardware, but at least in my case Wireshark turned out to work just fine.

And I suppose what you did is reverse engineering the driver by looking at the type of data that the wheel transfers when plugged.
You had to use Windows to perform these tasks, right? Because I'm led to think that you need to study the device when it functioning, to reverse engineer it.

Yep, spun up a Windows virtual machine and looked at the USB traffic with Wireshark on the Linux side. I did also try running Wireshark within the Windows VM, but that had some issues that I can't really remember the details of right now. That was a while ago though, things might work better these days.

[...] still didn't really put it into practice though, as I have to fight this very bad habit of thinking too big at the wrong times, ahah! [...]
[...] I see this happening in real time when I am coding, where I work in too many areas at once and end up with no concrete, testable code at the end of the session. [...]
[...] Yet I still have trouble to start such tasks; [...]

I would guess that pretty much anyone you speak to shares at least some of those concerns, they're really rather universal.

I've been mulling over what I want to write for about half an hour, but I guess you should just try to enjoy programming without worrying too much about the final result. I know it's a bit of a nothingburger, 'lol just do better', but you get better with practice, and you get practice by doing things. If thinking about big picture stuff paralyses you, you don't get any practice. Bit of a vicious cycle.

[anto31ad]

I guess you should just try to enjoy programming without worrying too much about the final result. I know it's a bit of a nothingburger, 'lol just do better', but you get better with practice, and you get practice by doing things. If thinking about big picture stuff paralyses you, you don't get any practice. Bit of a vicious cycle.

I think that the hardest part is really choosing what to work on. Could be the project itself, or a specific aspect of it.
So I believe the only way out of this vicious cycle, is to change the way a look at a problem I need to solve. This is where your initial advice is key: put the puzzle together piece to piece, with the pieces being as small as they need to be and to just enjoy this slow process.

I was able to do this the other day, even if on a very small scale:
I first asked myself how does a physic engine handle updates (or deltatime); So I just sat and thought about it and ended up coding a simple C program that just uses nanosleep every n milliseconds to mock the update calls; it's obviously a useless program, but it models one, very small, part of a physic engine;
Moreover, it got me to read documentation, learn about libraries and try out solutions;
Now I can take another small step like I just did, if I wanted to;
something that you can't always foresee beforehand is that if I ever decided that I don't want to work on this project anymore, I won't necessarily throw away the knowledge I gathered; which is why I think I should really develop this habit.

Anyway thanks for the Wireshark refs and the other tips you've provided, I will let you know if I ever get started with the T-LCM driver.
Right now I am pushing to put an end to the CS bachelor degree journey asap, so I will kinda have my hands almost full for a while.