Mac users can just sit right next to their Windows friends, get the Kinect data from the server and develop their own KinectVR games with Unity. Multiple mobile devices (both Android and iOS) are able to run at the same time. We encourage VR developers and Unity lovers to create their own experiences using this easy-to-use Unity plugin. Hello, I have a Kinect that I tried to plug on my Android device (Tronsmart T428), but it is not recognized. Is there a way to install the Kinect microphone drivers on an Android device? My goal is to create a Jarvis like domotic module that.
Depending on which model you have, the connection methods to your Android smartphone or tablet will be different. Microsoft Xbox 360 Wireless Controller Microsoft Xbox 360 Wired Controller.
Kinect for Xbox 360, or simply Kinect, is a motion sensing input device by Microsoft for the Xbox 360 video game console.
Based upon a webcam-style add-on peripheral for the Xbox 360 console, it enables users to control and interact with the Xbox 360 without the need to touch a game controller, through a natural user interface using gestures and spoken commands.This page has the download links for each version of the Azure Kinect Sensor SDK. The installer provides all of the needed files to develop for the Azure Kinect. Azure Kinect Sensor SDK contents. Headers and libraries to build an application using the Azure Kinect DK. Redistributable DLLs needed by applications using the Azure Kinect DK. With a Kinect and some software, you can make a decent 3D model of just about anything! Read on for our selection of the best Kinect 3D scanning software!
The second demo shows how a robot car can be controlledby gestures. The screen on the right shows the gesturesto control the robot car.
We will use an open source driver provided by OpenNI.
Plesae download the source file at OpenNI stable master tree
Please see ReadMe file(inside downloaded files) for detailed install instructions. - Alternative way to install under Windows
- Download Driver, OpenNI and PrimeSense files from cs1530 Google Code download page
- Unzip the zip file and install driver under 'BIN' folder
- Install OpenNI and PrimeSense respectively (Please use '0KOIk2JeIBYClPWVnMoRKn5cdY4=' as key when prompted)
(Since you are taking advantage of work done by CS1530students in the past, this approach will save you alot of time. However the installed version may notbe the most up-to-date.)
Currently Kinect supports Windows, Linux, Mac, Android.
(Since you must start from scratch to install everything,this approach is the most labor-intensive and prone to mistakes. However, once you are done you have installed the most up-to-date version.)(You can also take advantage of work done by CS1530students in the past, to install a version under Linux.Again, the version may not be up-to-date.)
then compile and run sample programs contained in the installation package (under OpenNI/Sample and OpenNI/platform/YOURSYSTEM/build/sample)
For Visual Studio user, there are precompiled VS project in OpenNI/platform/Win32/build/sample
Frequent Error Messages
'Failed to open XML, Error: Unknown USB Device Speed'
You need to use the power adapter for the Kinect camera when using a regular USB port on a PC
'error initializing NITE'
NITE isn't installed properly. Make sure you have used the correct license key. When it requests for the license key use the one provided by Primesense : 0KOIk2JeIBYClPWVnMoRKn5cdY4=
'Can't create any node of the requested type!'
NITE isn't installed properly. Make sure you have used the correct license key provided
I keep getting a message telling me I need .NET Framework 4.0.
This error can be fixed by installing with the .exe installer first then reinstalling with the .msi installer. If you use the auto-installer above you should not see this error.
'Kinect Motor Driver not found. LED & Motor control disabled'
You need to manually install the motor/ LED driver for the Kinect device.
How to use Kinect Interface?
After initial calibration, Kinect is able to track human movement in the form of match-stick skeletons.
The OpenNI Inteface will produce the position of each body part of interest (head, hand, elbow, etc.) in the form of its coordinates (x,y,z). In what follows,updateBodyInfo() pushes the device to grab next frame from camera andupdate BodyInfo in kinect object, andgetBodyInfo() retrieves the BodyInfo from kinect object.
A typical main program may look like the following:
The above program produces the following output (the coordinates of successive right hand positions):
All source code are saved on google code webpage (See next section for download instruction)
To develop components based upon the Kinect interface, you need to include BodyInfo.h/cpp, Kinect.h/.cpp, KinectFunction.h and Setup_XML.xml.
Also, make sure your program link to libraries under 'lib' folder and add 'include' as additional include folder.
Java Wrapper is also available on 'Kinect Java Wrapper'. The whole package is an Eclipse sample project.
Where to download the source code?
We will use google code for version control. These are detailed instructions on how to access google code svc server.Our course google code url is 'https://cs1530-kinect.googlecode.com/svn/trunk/'
How to provide an add-on Kinect Interface?
For some applications it is often more convenient tohave both a web interface and an add-on Kinect Interfacethat provides some of the functions of the web interface.Therefore a user can use the Kinect Interface forcertain essential functions and fall back on theweb interface for other functions.For the following example, the source code is available on google code:'http://code.google.com/p/cs1530-kinect/downloads/list'A video demo can be found on YouTube: 'http://www.youtube.com/watch?v=nmFo8dTNghw'
In the demo, after the Kinect calibration phase a 'start' gesture (a wave of the right hand) will startthe application. A 'stop' gesture (a horizontal left hand pointing ata vertical right hand) will stop the application.
When the GestureDetector in the main program HHAMain.cpp detects a gesturesuch as the 'start' hand gesture, the sendCommand(ICMD_START)outputs 'HOME HEALTH APPLICATION START' and starts the application.When the 'stop' gesture is detected the sendCommand(ICMD_STOP)stops the application and outputs 'HOME HEALTH APPLICATION STOP'.
The user can also use the web interface to dothe same. When the user clicks on the 'Start Application' button onthe web page test1.html, theassociated JavaScript function runCmd('start') (see the source for test1.html) sends a 'start' message toc:Program FilesHomeHealthAppHHARemoteClient.exe.The HHARemoteClient sends ICMD_START througha socket to the main program.The main program HHAMain maintains three threads tointeract with Kinect device, display device and HHARemoteClient.Upon reception of ICMD_START, sendCommand(ICMD_START) in the main program outputs 'HOME HEALTH APPLICATION START' and starts the application.It is the same for the 'stop' command.Please note the web page for this example only works properly on IE brower.
The application developer can add more buttonsto the web page test1.html, more commands to HHARemoteClient.cpp, and more processing functions to sendCommand() fora realistic application. GestureDetector() must alsobe expanded to detect additional hand gestures and body gestures.For this purpose a structure calledBodyInfo containing the 3D-coordinates of all body parts that can be tracked by Kinect is provided.The program is BodyInfo.cpp,and the corresponding structure is BodyInfo.h.More information can be found in the following Readme.txt.
Notice: MediaWiki has been updated. Report any rough edges to marcan@marcan.st
Here's what we know about the USB devices.
- 4Xbox NUI Motor
- 4.3Confirmed Information
- 5Xbox NUI Audio
- 6Xbox NUI Camera
Devices
Kinect has the following USB devices connected through its USB cable:
- Generic USB Hub
- Xbox NUI Audio
- Xbox NUI Motor
- Xbox NUI Camera
The devices do not conform to any standard USB class such as HID, camera, or audio devices. This indicates that using it as a plug-and-play webcam or microphone is probably not possible.
Plugging in Kinect
When you plug the Kinect to a PC running Windows, it finds the Generic USB Hub and the Xbox NUI Motor. The Motor obviously does not have drivers, so the PnP configuration fails and the device is not started. At this point, the Kinect still looks to be powered off.
Josh Blake tested creating boilerplate drivers using a Jungo WinDriver USB trial. When he generated and installed the driver for the Xbox NUI Motor, the following happened:
- The Kinect device started flashing green on the indicator light.
- The Xbox NUI Audio and Xbox NUI Camera devices became available on the USB hub (also with no drivers initially)
The flashing green light is consistent with the standard Kinect startup process. At this point it is waiting for the Xbox to boot up and probably for a handshake protocol or other command to activate the devices.
It appears that starting the Motor device also controls power to the rest of Kinect.
After the Audio and Camera devices were powered on, Josh also generated boilerplate drivers for them.
Talking to Kinect
Josh was able to send standard USB requests and get appropriate responses from all three devices.
He attempted to send vendor-type setup packets with various values for other parameters but it was mostly non-productive. There were a few interesting bits, documented below.
Until analysis is done on the USB protocol exchange between the Xbox and Kinect, controlling or getting data from Kinect is likely not possible.
Xbox NUI Motor
The Motor device controls the actual motor for tilting/panning the Kinect. It also seems to control power to the rest of the Kinect devices. After the driver for the Motor is installed/activated, Kinect displays a flashing green light and the other devices connect to the internal USB hub.
Endpoints
The Motor has only a single control endpoint on 0x00.
Data Experiments
Sending a vendor-specific setup packet read request (0xC2) with zeros for the request, value, and index and 0x02 for the length returns 0x80. This might indicate the current position of the motor as 0x80 is the middle of possible values (0x00 to 0xFF) and would be a good initial position for the motor.
Sending a vendor-specific setup packet write request (0x02) with a request of 0x00 and zeros for the value, index, and length resulted in (need to double check but it was a no-op, as opposed to an error)
Sending a vendor-specific setup packet write request (0x02) with a request of 0x01 and zeros for the value, index, and length resulted in the Motor device immediately disconnecting from the USB port, effectively resetting it.
Confirmed Information
The joint range of the Kinect motor is +31 degrees (up) and -31 degrees (down).
Setting Joint Position
The joint position can be set by sending:
Setting LED
The led can be set by sending:
where the led_options are enumerated as follows:
Reading Joint State
The joint state information is grouped in with the accelerometer data and is stored in the 1st, 8th and 9th byte of the return from:
the 1st byte (buf[1]) yields:
the 8th byte (buf[8]) yields:
the 9th byte (buf[9]) yields the following status codes:
Reading Accelerometer
The accelerometer data is stored in two byte pairs for x,y, and z:
Kinect Android Driver
The Accelerometer documentation (http://www.kionix.com/Product%20Sheets/KXSD9%20Product%20Brief.pdf) states there are 819 counts/g
Xbox NUI Audio
The Audio device provides combined audio from Kinect's four microphones. It also provides hardware noise cancellation by subtracting the TV's game audio and accounting for the 3D layout of the room.
Endpoints
The Audio device has several endpoints, the purpose of which is not yet clear. They provide enough bandwidth for full audio both in and out of the device. This supports the hardware noise processor and feeding game audio to Kinect for processing purposes only.
In addition to the default 0x00 control endpoint, the Audio device has:
- Endpoint 0x81 - IN, Bulk, 512 bytes
- Endpoint 0x01 - OUT, Bulk, 512 bytes
- Endpoint 0x82 - IN, Isochronous, 524 bytes
- Endpoint 0x02 - OUT, Isochronous, 72 bytes
Note that IN and OUT are relative to the USB host. OUT means sending data from the Xbox (or your computer) to Kinect.
Xbox NUI Camera
The most interesting and useful device, the Camera provides both an RGB and a depth map image.
Endpoints
The Camera device has two endpoints. Presumably, one of these provide the RGB image data and the other provides depth map data.
In addition to the default 0x00 control endpoint, the Camera device has:
- Endpoint 0x81 - IN, Isochronous, 1920 bytes (color camera)
- Endpoint 0x82 - IN, Isochronous, 1760 bytes (depth camera)
Note that the packet size is not large enough to fit an entire image at either 320x240 or 640x480, so a single frame will have to be transmitted and reconstituted from multiple frames. This also implies there may be some packet headers to help combine the image.
Interestingly, 1920 divided by 320 is 6, so without a header a single packet could contain 3 16-bit rows, and 80 packets would be required for a single 320x240 16-bit frame.
Kinect App
Data Experiments
Sending any vendor-specific setup packets, both read (0xC2) or write (0x02) results in a timeout, requiring resetting the device.
Driver Kinect Android App
Josh suspects that a handshake protocol has to occur, possibly with the Motor device, before the other devices will respond. Alternately, the camera might respond to a class setup packet or a non-standard setup packet type.
Comments are closed.