Analysis that cannot be displayed on my android Development Board by Kinect

We found on the internet that hackers have transplanted the Kinect to Android and used the beagleboard platform. The company wants to work hard on the Kinect, so it needs to implement the Kinect on the company's Android Development Board. Record for memo.

 

1. Follow the hacker's tutorial to transplant it to the beagleboard C4 platform. It went quite well. However, the deep image can only be maintained for several seconds. Because beagleboard is not my ultimate goal, this issue is put on hold and beagleboard is used as a reference. The hacker's tutorial is as follows:

Http://www.noritsuna.com/archives/2011/01/openframeworks_kinect_android.html

By the way, it is very troublesome to configure development tools, especially when some of them need to go out of the wall. Therefore, we recommend that you download the complete software package provided by the author at the end of the web page. Decompress the package and modify the environment variables. You can be more lazy: create a user name that is the same as the author, decompress the package to the corresponding folder, and the entire project and its software can be used.

 

2. Port it to your android Development Board. The hardware platform is not specific. It is not an international manufacturer.

Error: the accelerometer is displayed normally, but the depth image is not displayed.

So far, I have not carefully read the code. Haha. No way. Check the code.

 

Framework: openframework

Process: openframework> ofxandroid> ofxkinect> freenect> Linux kernel after libusb?

You can use the printed log for analysis. The reason is that ofxkinect: threadedfunction () function,

 

Void ofxkinect: threadedfunction () {</P> <p> freenect_set_led (kinectdevice, led_green); <br/> Reset (kinectdevice, freenect_depth_11bit ); <br/> freenect_set_depth_callback (kinectdevice, & grabdepthframe); </P> <p> oflog (of_log_verbose, "Leon ofxkinect: connection opened "); </P> <p> freenect_start_depth (kinectdevice); <br/> ............... <br/>}

The last statement sets the callback function. The implementation is as follows: Void freenect_set_depth_callback (freenect_device * Dev, freenect_depth_cb CB) <br/>{< br/> Dev-> depth_cb = CB; <br/>}Assign the callback function to Dev-> depth-CB, and then call freenect_start_depth (kinectdevice ); Int freenect_start_depth (freenect_device * Dev) <br/>{< br/> freenect_context * CTX = Dev-> parent; <br/> int res; </P> <p> .......................... <br/> Dev-> depth. pkt_size = depth_pktdsize; <br/> Dev-> depth. synced = 0; <br/> Dev-> depth. flag = 0x70; <br/> Dev-> depth. valid_frames = 0; <br/> res = fnusb_start_iso (& Dev-> usb_cam, & Dev-> depth_isoc, depth_process, 0x82, num_xfers, pkts_per_xfer, limit ); <br/> ................ <br/>}The function fnusb_start_iso () sets the depth_process parameter as the callback function, Fnusb_start_iso () {<br/> STRM-> cb = CB; <br/> libusb_fill_iso_transfer (STRM-> xfers [I], Dev-> Dev, EP, bufp, pkts * Len, Pkts, iso_callback, STRM, 0); <br/>}// Set the iso_callback function as the callback function in this function. Static inline void Merge (struct libusb_transfer * Transfer, <br/> libusb_device_handle * dev_handle, unsigned char endpoint, <br/> unsigned char * buffer, int length, int num_iso_packets, <br/> libusb_transfer_cb_fn callback, void * user_data, unsigned int timeout) <br/>{< br/> transfer-> dev_handle = dev_handle; <br/> transfer-> endpoint = endpoint; <br/> transfer-> type = libusb_transfer_type_isochronous; <br/> transfer-> timeout = timeout; <br/> transfer-> buffer = buffer; <br/> transfer-> length = length; <br/> transfer-> num_iso_packets = num_iso_packets; <br/> transfer-> user_data = user_data; <br/> transfer-> callback = callback; <br/>}The iso_callback function calls STRM-> CB and depth_process. Static void iso_callback (struct libusb_transfer * xfer) <br/>{< br/> int I; <br/> fnusb_isoc_stream * STRM = xfer-> user_data; <br/> If (STRM-> dead) {<br/> freenect_context * CTX = STRM-> parent; <br/> STRM-> dead_xfers ++; <br/> fn_spew ("EP % 02x transfer complete, % d left/N", xfer-> endpoint, STRM-> num_xfers-STRM-> dead_xfers); <br/> return; <br/>}< br/> If (xfer-> Status = libusb_transfer_completed) {<br/> uint8_t * Buf = (void *) xfer-> buffer; <br/> for (I = 0; I <STRM-> Pkts; I ++) {<br/> STRM-> CB (STRM-> parent, Buf, xfer-> iso_packet_desc [I]. actual_length); <br/> BUF + = STRM-> Len; <br/>}< br/> libusb_submit_transfer (xfer ); <br/>} else {<br/> freenect_context * CTX = STRM-> parent; <br/> fn_warning ("isochronous transfer error: % d/N ", xfer-> status); <br/> STRM-> dead_xfers ++; <br/>}< br/>}In depth_process, Dev-> depth-CB is executed. It is really troublesome to adjust it.

 

So far, the entire process has ended. When will the callback function be called? That is, where will transfer-> callback in the libusb_fill_iso_transfer function be called and executed? In ofxkinect, use cskinect to search for callback. Only the usbi_handle_transfer_completion callback uses transfer-> callback. Where will usbi_handle_transfer_completion be called? Search again

 

 

There are eight or nine items. We only care about the libusb_transfer_completed flag. There is only one, handle_iso_completion function. Search handle_iso_completion

 

Reap_for_handle --- op_handle_events. This function is set in the const struct usbi_ OS _backend linux_usbfs_backend struct. The value assignment statement is. handle_events = op_handle_events, Const struct usbi_ OS _backend linux_usbfs_backend ={< br/>. name = "Linux usbfs", <br/>. init = op_init, <br/>. exit = NULL, <br/>. get_device_list = op_get_device_list, <br/>. get_device_descriptor = op_get_device_descriptor, <br/>. get_active_config_descriptor = op_get_active_config_descriptor, <br/>. get_config_descriptor = op_get_config_descriptor, <br/>. open = op_open, <br/>. close = op_close, <br/>. get_configuration = op_get_configuration, <br/>. set_configuration = op_set_configuration, <br/>. claim_interface = op_claim_interface, <br/>. release_interface = op_release_interface, <br/>. set_interface_altsetting = op_set_interface, <br/>. clear_halt = op_clear_halt, <br/>. reset_device = op_reset_device, <br/>. kernel_driver_active = op_kernel_driver_active, <br/>. detach_kernel_driver = op_detach_kernel_driver, <br/>. attach_kernel_driver = op_attach_kernel_driver, <br/>. destroy_device = op_destroy_device, <br/>. submit_transfer = op_submit_transfer, <br/>. cancel_transfer = op_cancel_transfer, <br/>. clear_transfer_priv = op_clear_transfer_priv, <br/>. handle_events = op_handle_events, <br/>. clock_gettime = op_clock_gettime, <br/> # ifdef usbi_timerfd_available <br/>. get_timerfd_clockid = op_get_timerfd_clockid, <br/> # endif <br/>. device_priv_size = sizeof (struct linux_device_priv), <br/>. device_handle_priv_size = sizeof (struct linux_device_handle_priv), <br/>. transfer_priv_size = sizeof (struct linux_transfer_priv), <br/>. add_iso_packet_size = 0, <br/> };Check if linux_usbfs_backend is referenced, and where is. handle_events called? In libs/libusb/CORE. c Const struct usbi_ OS _backend * const usbi_backend = & linux_usbfs_backend; 

 

In libs/libusb/Io. function handle_events () {usbi_backend-> handle_events} io in C. C has two calls to the handle_events () function 4 Io. c libusb_handle_events_timeout 1946 r = handle_events (CTX, & poll_timeout); 5 IO. c libusb_handle_events_locked 2022 return handle_events (CTX, & poll_timeout); the correct call should be the second, libusb_handle_events_locked, search libusb_handle_events_locked, except that the function defines only one header file statement. It seems that the above analysis is incorrect. Is it libusb_handle_events_timeout that is called correctly? Libusb_handle_events_timeout is Api_exported int libusb_handle_events (libusb_context * CTX) <br/>{< br/> struct timeval TV; <br/> TV. TV _sec = 60; <br/> TV. TV _usec = 0; <br/> return libusb_handle_events_timeout (CTX, & TV); <br/>}Call, search for libusb_handle_events, and exclude the call by the bulk and control interfaces. Only one usb_libusb10.c fnusb_process_events 66 return libusb_handle_events (CTX-> CTX) in the two parts of libusb10.c ); 2 usb_libusb10.c fnusb_stop_iso 245 libusb_handle_events (CTX-> USB. CTX); 2 is the call when the USB is stopped, and 1 is our goal, Int fnusb_process_events (fnusb_ctx * CTX) <br/>{< br/> return libusb_handle_events (CTX-> CTX); <br/>}We have seen fn. Oh, our path should be correct. Int freenect_process_events (freenect_context * CTX) <br/>{< br/> return fnusb_process_events (& CTX-> USB); <br/>}Search freenect_process_events, Libs/libfreenect. HPP Template <class T> class freenect: noncopyable {<br/>... <br/> // do not call directly, thread runs here <br/> void operator () {<br/> while (! M_stop) {<br/> If (freenect_process_events (m_ctx )! = 0) Throw STD: runtime_error ("cannot process freenect events"); <br/>}< br/>... <br/>}// Do not call directly, thread runs here Haha, finally arrived.

 

 

 

The whole process is like this, but there is no solution to problems that cannot be displayed. Log is required. First, print the libusb log. No log is like a blind man.

 

 

 

 

On November 4.15, the editor output the libusb log and wrote it as a memo. The libusb log part is actually defined in libusb. h. Enum usbi_log_level {<br/> log_level_debug, <br/> log_level_info, <br/> log_level_warning, <br/> log_level_error, <br/> }; <br/> void usbi_log (struct libusb_context * CTX, Enum usbi_log_level, <br/> const char * function, const char * format ,...); <br/> void usb_oflog (INT loglevel, const char * format ,...); <br/> # ifdef enable_logging <br/> # DEFINE _ usbi_log (CTX, level, FMT ...) usbi_log (CTX, level, _ FUNCTION __, FMT) <br/> # else <br/> # DEFINE _ usbi_log (CTX, level, FMT ...) <br/> # endif </P> <p> # ifdef enable_debug_logging <br/> # define usbi_dbg (FMT ...) _ usbi_log (null, log_level_debug, FMT) <br/> # else <br/> # define usbi_dbg (FMT ...) <br/> # endif <br/> # define usbi_info (CTX, FMT ...) _ usbi_log (CTX, log_level_info, FMT) <br/> # define usbi_warn (CTX, FMT ...) _ usbi_log (CTX, log_level_warning, FMT) <br/> # define usbi_err (CTX, FMT ...) _ usbi_log (CTX, log_level_error, FMT) <br/>Finally, all log outputs are located in the usb_oflog (Level, FMT) function, in core. c Void usbi_log (struct libusb_context * CTX, Enum usbi_log_level level, <br/> const char * function, const char * format ,...) <br/>{< br/> va_list ARGs; <br/> file * stream = stdout; <br/> const char * prefix; <br/> # ifndef enable_debug_logging <br/> usbi_get_context (CTX); <br/> If (! CTX-> Debug) <br/> return; <br/> If (Level = log_level_warning & CTX-> debug <2) <br/> return; <br/> If (Level = log_level_info & CTX-> debug <3) <br/> return; <br/> # endif <br/> switch (level) {<br/> case log_level_info: <br/> prefix = "info"; <br/> break; <br/> case log_level_warning: <br/> stream = stderr; <br/> prefix = "warning"; <br/> break; <br/> case log_level_error: <br/> stream = stderr; <br/> prefix = "error"; <br/> break; <br/> case log_level_debug: <br/> stream = stderr; <br/> prefix = "debug"; <br/> break; <br/> default: <br/> stream = stderr; <br/> prefix = "unknown "; <br/> break; <br/>}< br/> fprintf (stream, "libusb: % s [% s]", prefix, function ); <br/> va_start (ARGs, format); <br/> vfprintf (stream, format, argS); <br/> va_end (ARGs ); <br/> fprintf (stream, "/N"); <br/>}< br/>Libusb log output is a standard input output. There is a solution to save logs to a file in the above function. I use another method: Android log Mode 1. Modify libusb. h Enum usbi_log_level {<br/> log_level_debug = 3, <br/> log_level_info, <br/> log_level_warning, <br/> log_level_error, <br/> }; </P> <p> void usbi_log (struct libusb_context * CTX, Enum usbi_log_level, <br/> const char * function, const char * format ,...); <br/> void usb_oflog (INT loglevel, const char * format ,...); <br/> # ifdef enable_logging <br/> # DEFINE _ usbi_log (CTX, level, FMT ...) usb_oflog (Level, FMT) <br/> # else <br/> # DEFINE _ usbi_log (CTX, level, FMT ...) </P> <p> # endif </P> <p> # ifdef enable_debug_logging <br/> # define usbi_dbg (FMT ...) _ usbi_log (null, log_level_debug, FMT) <br/> # else <br/> # define usbi_dbg (FMT ...) <br/> # endif <br/> # define usbi_info (CTX, FMT ...) _ usbi_log (CTX, log_level_info, FMT) <br/> # define usbi_warn (CTX, FMT ...) _ usbi_log (CTX, log_level_warning, FMT) <br/> # define usbi_err (CTX, FMT ...) _ usbi_log (CTX, log_level_error, FMT) <br/>The above Enum is used to assign the first value to 3, and relocate several macro definitions to another function usb_oflog. This function needs to be modified by ourselves. Add the following at the end of core. C: # Include <Android/log. h> <br/> # define log_tag "of" <br/> # define lognotice (...) _ android_log_print (android_log_info, log_tag ,__ va_args _) <br/> # define logwarning (...) _ android_log_print (android_log_warn, log_tag ,__ va_args _) <br/> # define logerror (...) _ android_log_print (android_log_error, log_tag ,__ va_args _) <br/> # define logfatal (...) _ android_log_print (android_log_error, log_tag ,__ va_args _) <br/> # Define logverbose (...) _ android_log_print (android_log_info, log_tag ,__ va_args _) <br/> // # define vlognotice (...) _ android_log_vprint (android_log_error, log_tag ,__ va_args _) <br/> // # define vlogwarning (...) _ android_log_vprint (android_log_error, log_tag ,__ va_args _) <br/> // # define vlogerror (...) _ android_log_vprint (android_log_error, log_tag ,__ va_args _) <br/> // # define vlogfatal (...) _ android_log_v Print (android_log_error, log_tag ,__ va_args _) <br/> // # define vlogverbose (...) _ android_log_vprint (android_log_error, log_tag ,__ va_args _) <br/> void usb_oflog (INT loglevel, const char * format ,...) {<br/> // thanks Stefan! <Br/> // http://www.ozzu.com/cpp-tutorials/tutorial-writing-custom-printf-wrapper-function-t89166.html <br/> // If (loglevel> log_level_info) {<br/> if (1) {<br/> va_list ARGs; <br/> va_start (ARGs, format); <br/> If (loglevel = log_level_info) {<br/> logverbose (format ); <br/>}< br/> else if (loglevel = log_level_debug) {<br/> lognotice (format ); <br/>}< br/> else if (loglevel = log_level_warning) {<br/> logwarning (format ); <br/>}< br/> else if (loglevel = log_level_error) {<br/> logerror (format ); <br/>}< br/> va_end (ARGs); <br/>}< br/>See the android log rule reference. Then re-compile and you will be able to see the cute logs.

 

 

Updated on November 4.26

 

You have already run on your own Development Board. There is an image output, but it is only a two-value graph. This should be a problem when drawing on the upper layer. Record the solution process first.

After the libusb log is successfully printed, the analysis shows that the data obtained by libusb is 0, and the underlying support of libusb is usbfs. Therefore, the USB/CORE/devio section of the kernel is tracked. C contains processcompl (),

Convert statements

If (AS-> userbuffer & urb-> actual_length ){

After the image is changed to if (as-> userbuffer) {, the image can be output normally. Contact the supplier of the board. urb-> actual_length is not assigned a value in the USB core. Bug. In driver/XXX/USB/dwc_otg/dwc_otg_hcd_intr.c, if (++ _ QTd-> isoc_frame_index = urb-> number_of_packets) {Add the following statement urb-> actual_length + = frame_desc-> actual_length. The final result is: Default: dwc_error ("% s: unhandled _ halt_status (% d) /n ", _ FUNC __, _ halt_status); bug (); break;} urb-> actual_length + = frame_desc-> actual_length; if (++ _ QTd-> isoc_frame_index = urb-> number_of_packets ){

 

The frame rate is very low if it is changed based on the solution provided by Alibaba Cloud. The reason is to be analyzed.