Changing the array and Leap Motion® camera module positions
All Ultrahaptics development kits contain a Leap Motion® camera module for tracking the position of the hand and is mounted in a cradle attached to the array. Our demo software will automatically detect the array type and configure the alignment necessary to map between the tracking camera and array.
Ultrahaptics SDK comes with example code, showing how to use the data provided by the tracking device and control the array.
Our API provides the Ultrahaptics::Alignment class as the mechanism for mapping between each. The alignment object can either be initialised by reading the device information, set using one of the provided library options, or overridden to use your own customised tracking position, either via an XML configuration file or directly in the constructor.
Alignment tutorial
NOTE: Any modifications of camera position apply only to our development kits and not the Ultrahaptics STRATOS Inspire. The Ultrahaptics STRATOS Inspire is a fully integrated device and should not be disassembled.
This tutorial will explain in more detail the matrix algebra used to map between each and work through examples to show how you can define your own tracking/array mapping.
The Leap Motion® camera module and Ultrahaptics arrays use different coordinate systems. Ultrahaptics array operates in metres, while the Leap Motion® camera module returns values in millimetres.
The Ultrahaptics system uses the following units:
- Distance: metres
- Frequency: Hertz
- Modulation Amplitude: arbitrary unit varying from 0 to 1
Coordinate System
The Ultrahaptics system employs a right-handed coordinate system. As the device sits on a desk with the status LED facing towards you, the X and Y axes lie in the horizontal plane. The origin is centred on the surface of the transducers in the centre of the device (ignoring the Leap Motion® camera module). The X-axis runs left to right, with increasing positive values to the right and negative values to the left. The Y-axis runs bottom to top. The Z-axis is vertical with positive values increasing upwards. As the origin is on the surface of the device, there cannot be a negative Z-value.
Ultrahaptics array coordinates
Tracking System
Many of the best uses of the Ultrahaptics technology are enabled when combined with a tracking system to detect the actions and location of the user. Any tracking system can be used; Ultrahaptics SDK only needs the location and intensity of the haptic points it creates.
Ultrahaptics development kits are supplied with a Leap Motion® camera module that is held in a pre-calibrated position. Like the Ultrahaptics system, the Leap Motion® camera module also uses a right-handed coordinate system. However it is rotated and translated so that the origin is at the Leap Motion® camera module as shown below.
Leap Motion® coordinates
Alignment class
To map between the two, the Alignment class uses a 4×4 mapping matrix. For example, the TOUCH Development kit mapping is achieved by setting up the Alignment object with an included library XML file:
Ultrahaptics::Alignment alignObj("U5_square.alignment.xml");
Internally, this uses the following matrices operations:
Where (xd, yd, zd) is the coordinate in the Ultrahaptics device (array) space and (xt, yt, zt) is the coordinate in the tracking space. The 110.5 value in the fourth column of the matrix in the first equation is the distance along the tracking device’s Z-coordinate from the centre of the array.
Translation
Moving the tracking device to a different position around the array, i.e. in the array’s XY-plane, without changing its orientation is simply a matter of manipulating the fourth column of the above arrays. The first example (on the left) shows how we can place the tracking device 100mm to the right of its default position (case 1) and the second example (on the right), centred on the right-hand side 200mm from the array origin (case 2):
Here, we show the device-to-tracking matrix M, since the Alignment class can be set explicitly using the first three rows of this (remember, the tracking-to-device matrix is the inverse of this):
Case 1
Case 2
The fourth column can be summarised as containing the coordinate of the centre of the array in tracking space in millimetres, prior to any rotation. For Case 1, the centre of the array is {-100, 0, 110.5}, for Case 2, the centre of the array is {-200, 0, 0} from the Leap Motion® camera module origin.
Rotation
Plane rotation can be achieved by applying a rotation matrix. A rotation by an angle θ, where θ is the rotation from tracking device to array, about the x-, y- or z-axis can be achieved by inserting the sub-matrices below into a 4×4 identity matrix and pre-multiplying the translation matrix:
Note that the TOUCH Development Kit alignment matrix itself applies a rotation, since we rotate along the tracking device’s YZ plane.
Rotation by 90°
If the Leap Motion® camera module is rotated by 90° clockwise at the cradle position, a rotation of -90° in the Leap Motion® camera module’s XZ plane (shown below), we pre-multiply the original device-to-tracking transformation by Ry(θ=-90°).
Rotation and translation
We can combine the two operations to place the tracking device anywhere around the array. Here, the tracking device is placed on the right-hand side of the array, 120mm from the centre and rotated clockwise 90°.
Our rotation sub matrix is Ry(θ=-90°). The device-to-tracking transform equation is now:
Note that the fourth column of the second matrix above contains the coordinate of the centre of the array from the point-of-view of the Leap Motion® camera module prior to rotation:
{-120, 0, 0}
Rotation in the Leap Motion® camera module’s XY-plane – Tilt
In this example, our tracking device is mounted above and slightly closer to the centre of the array. You can see from the picture that face of the camera is tilted forward and so is no longer flush with the array.
Breaking this down, we define the offset coordinates for the array from the camera. This is measured as {0, -19, 95.5}, i.e. the origin of the Leap Motion® camera module is 19mm in front of the array and 95.5mm along the Z-axis.
This gives us our translation matrix:
The rotation happens around the Leap Motion® camera module’s x-axis, so that we must use the Rx(θ) rotation matrix. Placing a rule along the face of the camera and measuring its distance from the intersection of the array gives us the hypotenuse to calculate the angle: approximately -6°, (-0.105 Rad).
Our final equation looks like this:
Combining tilt, rotation and translation
Taking the example above and placing the Leap Motion® camera module along the right-hand-side of the array requires applying a rotation in the Leap Motion® camera module’s XZ-plane, followed by the tilt operation (note now that the Leap Motion® camera module is titled out and aligned in the opposite direction with the cable pointing down).
To do this, we create our translation matrix. Here, the translation vector is {-122.5, -6, 0}. We then pre-multiply by Rx(θ=-6°), followed by our tilt operation of . In abbreviated form:
The order of pre-multiplication is dependent on the steps needed to rotate the Leap Motion® camera module to its final position. For the example above and from right to left: translation, rotation, tilt. In the same way, we can even add tilt in the Leap Motion® camera module’s z-axis by applying Rz.
To change the alignment in your application at runtime, we recommend adding the alignment matrix and its inverse to an XML file that can be imported in your code. The file name is used as the argument in the Alignment object constructor:
Ultrahaptics::Alignment customAlignmentObject("/path/to/my/customAlignment.xml");
As an example, the XML alignment file for the default TOUCH Development Kit camera position looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<AlignmentTransforms>
<DeviceToTrackingSpace>
<Row x="1000.0" y="0.0" z="0.0" w="0.0" />
<Row x="0.0" y="0.0" z="1000.0" w="0.0" />
<Row x="0.0" y="-1000.0" z="0.0" w="110.5" />
<Row x="0.0" y="0.0" z="0.0" w="1.0" />
</DeviceToTrackingSpace>
<TrackingToDeviceSpace>
<Row x="0.001" y="0.0" z="0.0" w="0.0" />
<Row x="0.0" y="0.0" z="-0.001" w="0.1105" />
<Row x="0.0" y="0.001" z="0.0" w="0.0" />
<Row x="0.0" y="0.0" z="0.0" w="1.0" />
</TrackingToDeviceSpace>
</AlignmentTransforms>Simply calculate your alignment matrix and replace the above using the first three rows.
0 Comments