Hand module and DCS

Can somebody explain how i can make the hand module work in DCS? The video only shows it working but i like to know how i can get it to actually work.

Thanks

1 Like

Not yet, but since I finally have my hand tracking module, I hope to test it soon…

1 Like

Does the hand module allow you to operate controls and switches in DCS without a controller? Boy would I like that!
:slight_smile:

I have it and it works…but not in DCS…

1 Like

I am confident it can be done. I will report back when I have the full story.

3 Likes

Any luck?
Found this article from someone who apparently works with UltraLeap.
Apparently, to get the Leap Motion working with DCS, it requires mouse emulation “modules”. Found this one, but not sure anyone got it to work. Or this one from last year.

Not sure that I want to be installing unknown software, tho! The main take-away, for me, is that it’s not ‘native’ and would require a hack, not just changing some settings. The fact that someone from UltraLeap was talking about it back in February gives me a tiny bit of hope for the future, tho! If anyone follows them, poke 'em!

I am sure it can be done, but I have been trying to finish a Linux automation project first.

Installing some third party stuff is pretty much par for the course with DCS World. If it has been around a while, and doesn’t usually turn up any malware warnings, it is probably safe.

Those videos look very interesting. Thing about DCS World is, the hand controllers were always usable, it was the inconvenience of using them with HOTAS that was such a problem.

That sounds suspiciously like you’re putting your work in front of your play! For shame! :grin:

Here’s a video of someone using hand tracking with DCS:

Here’s the link to the controller emulation project:

YMWV

3 Likes

Thanks for weighing in, that helps. I will make sure and get this working.

Oh, and that video looks amazing. Controller emulation outright should be awesome.

My Linux automation project has VR uses.

Lol, you’re forgiven then! :grin:

@PimaxUSA thanks for that! This could be interesting!

Of course, using this requires that the user has actually setup the controllers in DCS, properly, in the first place! I have something setup but I don’t know if I ever bothered to set them up properly…

To anyone who looks at the github, “Opisthenar” is the back of your hand. So one of the gestures is to touch the back of one of your hands. That should be interesting! I’ve never seen that or the gesture where you touch the palm of your hand. Hmmm, I wonder if I can sub in “the middle finger” for “escape”… Probably not a good habit to get into anyway.

Just check the box ‘enable touch controllers’ or whatever in DCS World, and it should work. The gesture thing is probably referring to holding the hand upside down to indicate a right-click equivalent - though in practice you can just ‘press’ buttons with the index finger.

From what PimaxUSA has posted, I should not have much difficulty getting really great results from the hand tracking.

1 Like

Well, that was disappointing.
I’ve got a couple of main issues.
-In order to get my controllers to “activate” (green beam that I can point), I have to squeeze the grip. When I am “squeezing the grip”, I can’t really make any gestures except “squeezing the trigger”. Is there a setting or other way to address this? I’ve had to squeeze the grip on my OG Vive wands way back when and I have to do it now with my Knuckles.
EDIT: what I’m describing is what is required to interact with the gui, NOT the controls in the cockpit! For those, just reach out and touch them! This actually works ok for certain things, however, I’m attempting to fly a helicopter. Turning the throttle is… well, I think I still have a few strands of hair left. What I ended up doing, and it’s still not very functional, is to disable the gui setting for the Vive controller for left mouse click. That left me with a right click function but no confusion from the multiple clicking. And turning knobs with a mis-oriented hand is a real chore, which means that most of us will need to spend time tweaking the rotation settings (remember this needs to restart steamvr to check every adjustment). Even then, rotating knobs is iffy at best. Whatever this guy is using to do recognition sucks. Hard. Hands freeze constantly or move out of sync with your hands up to a foot away. Yeesh. Nice proof of concept but this ain’t usable for me without better accuracy.

-In the Readme on the Github, he mentions a number of different choices (mappings) for the gestures. Unfortunately, when I go to configure my controllers in DCS, my only options are “JOY_BTN_1” and 2, and 4 entries as “JOY_BTN_POV1”-4 for the touchpads. Before setting up the drivers, my choices were labeled “Primary” and “Secondary” and Primary.Trackpad1"-4. So no new choices, just a change of labels. But nothing is labeled as indicated in the Readme, nor do I have any of the other options. Do I have something set up wrong? If so, it goes back a long way as I’ve never had any other options.
EDIT: again, this is a gui related issue. I’ll do more checking on the gestures, I think that I’m using things wrong. Edit: it can be helpful to have the gui usable by controller, but I found that it just complicated things and caused multiple extra clicks. Map the gui mouse controlls to the hotas and leave the controller out of it. That leaves the controller able to touch controls without confusion. I did leave right click on the controller to access the throttle for my Huey. Doesn’t work well, tho.

Next, I found the alignment off a bit. Enough to make it difficult to point accurately. Forget twisting a knob! So I started messing around in the settings.xml file, with the “…HandOffsetTotation” values and found that they work with 3D rotational mathematics. That crap gives me a headache just knowing it exists! However, if you want to mess with it, you can get the computations from a website like this one and it will spit you out some numbers that you can use. However, you need to restart SteamVR for every adjustment, so it gets tedious very quickly.
I only messed with the right hand but the values that I came up with were x=+20 and y=+12 (or “rightHandOffsetRotation” value=“0.1733303 0.1039982 0.0 0.9793574”).

Now a couple of interesting things here! First, this is a SteamVR thing so you are actually changing the alignment of your controller in VR. This can be an issue for other games! We should be able to enable/disable the drivers by removing that one line in the steamvr.vrsettings file.
Second, with the additional drivers, you can now see your “hands” in VR! They’re skinned as controllers, obviously, but your hands now work! Sort of. At least there’s an input from your hands to work with now!
EDIT: actually, I find that controlling SteamVR with my hand gestures to work fairly well! I need more training, but the software works. Could definitely use some love to get accuracy up.

I played around a little with VR Toolbox, but I’ll need to play around with it more to see if I can get something usable. No pointer is a bit of an issue, but it’s a start!
Not being familiar with most of the gestures doesn’t help. Edit: nope. Inconsistent tracking and random poses makes this a real pia. The drivers will need to be ‘unplugged’ for this one! Bummer!

Last, but by no means best, I found that the consistency of the hand pose tracking absolutely sucked! My hand is constantly changing into and out of the “grip” pose and the trigger will actually twitch! Depending on how your hand is oriented makes it better or worse, but better is really lousy!
So, after getting everything else all set up, I found it unusable in it’s current state. There is a decent chance that it’s a lighting issue, so I will check again tonight, to see if it’s anymore stable. Edit: a little more stable at night but not by enough.

Btw, set up was easy, just follow the directions on the Readme. One thing that I did NOT do was to update my UltraLeap drivers to the latest beta Orion release but I’m a little scared about doing that. I don’t know if that would break anything for the Pimax module.
If anyone has any ideas, I’m up for trying them. That video shows a MUCH more stable experience than I had!

If “we” could get this set up properly, since our hands are being “seen” as Steam Controllers in SteamVR, we could actually use only our hands for a number of apps, leaving our controllers to gather dust! Hopefully my stability issues are something that I can fix!

Edit: recalibrated device for slight improvement but not enough to make any real difference.

Edit: while playing around with the handtracking over the last couple of months, I’ve found that there is a real need to be able to “pause” or stop the tracking. The system is a little too energetic in looking for “poses” and can/does activate things accidentally. I’m not sure how this would work for the general public because you need to be able to turn it on without taking off the headset. For the rest of us, something like a command line that we could use with VoiceAttack for would help.

That has not been my experience. According to my experience, while DCS will see your controllers, IF they are on at startup, you need to go the controller/gui section and set the mapping before the controllers will do anything. You can map your controller buttons to any of the other controls, but the gui section is the main one. In addition, no matter which control you are trying to map, you only have 10 choices. 1 for each trigger and one for each direction on each thumbpad. I can’t map ‘A’ or ‘B’ and ‘grip’ is pre-registered as ‘activate controller’. At least for me…

Note that we’re talking about DCS, I believe the other button mappings work in SteamVR, they’re just not seen in DCS. For me.

I never experienced having to do any of that, with the Vive Wands, the Index Controllers, or IIRC, even Oculus Touch controllers. Somehow, you don’t have to do any of that.

Maybe you have very old DCS World ‘savegame’ data that uses some older controller config scheme.

You might want to try using the DCS World OpenBeta though.

Interesting. I’m on the Steam version and I thought I was running the beta. Nope! Switching to the OpenBeta version, thanks! I’ve actually heard that we need to be running the beta version for performance as well. Guess I never got around to it. :thinking:
I’ll update a little later…

That may be another difference. Never even tried the Steam version, as the last thing I need with regard to DCS World seems to be yet something else in the loop to go wrong…

No change. Still only the 10 choices.
When you have some time, could you look at your DCS controller configuration and see if you have more/different options? Whether it’s automatic isn’t as big a deal as whether I’m missing options, imho.

However, I’m learning how to control SteamVR without controllers! This is cool! Takes a learning curve on how to position your hands so the gestures are recognized, but it’s working!

@SweViver could you look at this guy’s work? If you could get something like it to the accuracy of what you’ve gotten to with PE, it would be epic! What a score for Pimax that would be! Hand tracking in SteamVR, as well as PE!
Edit: Even better, reach out to him and see if he want’s to play!

Found another discrepancy. In the video, that guy is using proximity to activate the controls, not a gesture emulation. I have to crook my finger, he’s just poking things. I’ll be very interested in what you come up with when you have time to play with it.