First Person Driving with a Wheel

(Update 12/3/19: portions of this project are described under Patent US10493363B2)

I’ve hacked together a first person driving experience that uses a natural steering wheel to control an RC car. The experience is a lot like playing a Racing Kart game IRL. It’s probably easier to understand if you see it in motion. Check it out:

AlexDrive.gif

I got the idea for this project while watching my three-year-old son play with his radio controlled toy cars. These all use the conventional two joysticks. The left is isolated to vertical movement to control the motor and the right stick is isolated to horizontal movement to control the turning direction of the front wheels.

As a toddler, my son tends to only meaningfully control one stick at a time. It’s probably too sophisticated for him to infer that directing the car to a diagonal means he needs to press the left stick forward and combine that with the right stick’s sensitivity. What’s interesting, though, is that he has no such difficulty driving his big power wheel. Same operations but different interface. It has a natural steering wheel, a gas pedal, and a shifter to control direction. It got me thinking: what if RC car’s could be controlled by natural steering wheels too?

I quickly realized that having a more natural controller isn’t enough. Without locking my son’s orientation to the car’s, I would just be recreating the struggle I had with the arcade game Super Sprint. I could never tell if turning the wheel left was moving the avatar screen left/right/up/down. Luckily, with the rise of drone racing, I could simply mount an FPV camera and place the monitor as if it were a windshield or wear it directly on my face.


The combination of a natural steering wheel controller and visual feedback directly from the car itself makes for a really compelling experience. The perspective is really novel and the scale magnifies the intensity of speed. It can feel like travelling at 300 mph, but at the same time, everyone who has picked it up can deftly control the car within minutes. It’s a lot more approachable than a drone. Who knows, with autonomous cars on the horizon, maybe this is how the joy of driving is preserved?

I had a lot of fun making, breaking, and remaking this rig. I learned a lot about electronics in the process and it’s been really fun sharing the experience with friends, and now with you!

38 responses to “First Person Driving with a Wheel”

    • I found the NRF24L01 finnicky, especially if you are buying off-brand components. I suspect the pin outs are not always the same. Also be sure it gets a consistent 3.3V, a regulator may help there.

    • Sure! Just search for “FPV” (first person view) cameras and monitors. There are approximate 40 channels available on the 5.8GHz frequency for analog video. Match your camera transmitters channel to the receiver and you’ve got a live picture.

  1. Paul please, pass the code of the Arduino and one diagrama, its for one proyect of fair to science, for orientation, tanks
    I am from México, my English no is good

  2. Paul are you willing to share the code for this project? I have ordered all of the components already! It would be greatly appreciated.

  3. […] [Paul Yan] was watching his young son with a radio controlled toy, and was struck by how the two-joystick control layout is not necessarily as intuitive as it could be. By contrast when faced with a console game with first-person view and a steering wheel the boy had no problem dropping straight into play. This observation led him to investigate bringing a console steering wheel to an RC car, and the result is a rather impressive FPV immersive driving experience. […]

  4. Hello! First of all congratulations for this great project! I am looking into the first person view camera and monitor. Could you tell me how much you paid for them?

    Thank You

  5. Nice Job! How do you make the Arduino, PS2-Shield and the nRF work together? I use a USB-Shield for my XBox360 controller, but have problems with SPI-BUS and the library…

  6. Nice Job! How do you communicate between Uno, PS2-Shield and the NRF? I have problems with the SPI-BUS. I use the NRF24L01+ modules and have problems… when I use only the two Uno´s and the NRF the communication is OK, but with USB-Shield (I use a XBOX Controller) the SPI-BUS communication is in trouble…

  7. Congratulations this project its awesome!
    Can you please tell for how many metres you can control the car? And if you think that there a way to augment the area?

    • I have not tested the extends of the radio, but I have seen a few articles online that have done this. It will vary depending on the type of antenna and brand.

  8. Hi paul. Awesome work. I am making an attempt to replicate it. But i cant afford the FPV camera and the monitor. Instead I am having a wireless IP camera and an android application capable of streaming the video. I have tested it and it is found to have a slight delay (maybe half a second). considering this delay, will the system work fine?

  9. […] Das dachte sich der Designer Paul Yan, der dieses Projekt mit zwei Arduinos, einem alten Playstation 2-Lenkrad, einer Kamera und einer VR Brille realisiert hat. Die Idee dazu bekam er, als er seinem dreijährigen Sohn beim Spielen mit seinen ferngesteuerten Autos zusah. „Ich habe einfach eine Fahrerfahrung aus der Ich-Perspektive kreiert, bei der man ein Lenkrad zum Steuern benutzt. Diese Erfahrung erinnert sehr an das Spielen von Kart-Videospielen, allerdings im wirklichen Leben“, so Yan. Am besten seht ihr euch dieses Video auf Youtube an, um zu verstehen, wie cool Paul Yans Projekt ist! Stellt euch nur die Möglichkeiten vor, die einem die neue Perspektive bietet. Man kann unter parkenden Autos fahren, durch hohes Gras und und und! Einen ausführlichen (englischen) Beitrag dazu findet ihr in dem Blog des Designers: https://pauldyan.wordpress.com/2016/09/07/first-person-driving-with-a-wheel/ […]

  10. Hi Paul,

    Just a quick question, do you have a recommendation of somewhere I should read up about transmitting controlls to a remote controlled car? I’m thinking of doing something smiliar and thinking about how to deal with the car going out of range/losing connection to the “controller”.

    Do you do a sort of “are you still there?” check from the car to the receiver multiple times a second and if the car can’t “talk” to the controller anymore stop moving? Is this something you wrote yourself or is there some sort of protocol to follow. I know about networking protocol for transmitting data but I’ve only done it for sending packets of data rather that streaming things so I’m a bit unsure as to what to do.

    Any advice you feel comfortable sharing would be much appreciated. Great project by the way, I love it.

    Thanks in advance,

    Sean

    • Hi Sean,
      What you’re suggesting is smart and it’s something I didn’t get around to implementing but if I did, I would do as you describe – have the car check for a connection and zero out the throttle if there isn’t one. If you’re writing it from scratch like I did, I imagine you’d have to write this check yourself. Some people have suggested that you could hijack an existing RC transmitter/receiver and in that case you could rely on the checks that are already in place.

  11. I’m a big brother was really into RC stuff and about 7 years ago she was in an automobile accident she’s trying to get back into it but it’s really difficult because he only has use of one hand and the use of one foot this setup would work perfect can I could get some instructions or could you Point me too somewhere where there are instructions I would be forever grateful

    • Go to a hobby shop, buy an FPV camera setup and find a way to strap it to an RC and keep it powered. Buy an FPV goggle and you’re half of the way there. Serious RC people probably prefer the conventional pistol grip so you might save yourself a whole lot of trouble trying to wire up a wheel. Hope that helps

  12. […] Das dachte sich der Designer Paul Yan, der dieses Projekt mit zwei Arduinos, einem alten Playstation 2-Lenkrad, einer Kamera und einer VR Brille realisiert hat. Die Idee dazu bekam er, als er seinem dreijährigen Sohn beim Spielen mit seinen ferngesteuerten Autos zusah. „Ich habe einfach eine Fahrerfahrung aus der Ich-Perspektive kreiert, bei der man ein Lenkrad zum Steuern benutzt. Diese Erfahrung erinnert sehr an das Spielen von Kart-Videospielen, allerdings im wirklichen Leben“, so Yan. Am besten seht ihr euch dieses Video auf Youtube an, um zu verstehen, wie cool Paul Yans Projekt ist! Stellt euch nur die Möglichkeiten vor, die einem die neue Perspektive bietet. Man kann unter parkenden Autos fahren, durch hohes Gras und und und! https://www.youtube.com/watch?v=DCkqqOrIXCw Einen ausführlichen (englischen) Beitrag dazu findet ihr in dem Blog des Designers: https://pauldyan.wordpress.com/2016/09/07/first-person-driving-with-a-wheel/ […]

  13. Hey Paul. Three classmates and I are working on a project just like this for our senior design class for Computer Engineering. We are in the design stages and received all of our parts. We have a beginner grasp of the scope of the project, but I’m sure we will have many questions. I would like to get some contact information to ask you some questions later on in the future. We would appreciate your help.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: