FTDI Community

General Category => Discussion - Software => Topic started by: lzerman on April 08, 2021, 08:45:53 AM

Title: USB HOST UVC Camera FTDI Vinculum-II not reading video stream
Post by: lzerman on April 08, 2021, 08:45:53 AM
I have extracted the minimal program down to a single file necessary to demonstrate the issue I am having outside the context of the larger program. At the highest level of description the USB Host (FTDI Vinculum-II) connects to a UVC camera, properly enumerates the device interfaces, end points, etc., sets up for transfer, but in the end when it finally does a vos_dev_read() (line 511), it returns with no error, and zero bytes read.  The Host -> UVC camera ioctl r/w work fine and all values retrieved from the UVC camera (multiple cameras have been used in testing) are validated properly against the USB dumps from USB Device viewer. So, the bi-direction conversation is working great, just no video data. If I change values to invalid values or unplug the camera, I do get the expected error codes.

To be clear, this derivative program does nothing with the data read beyond it writes out to the UART terminal (9600 baud, no flow-control) the number of bytes read. Bottom line; find a fix for this program and the same will work for the real program.

I know it would be finding a Unicorn with someone who has experience with the Vinculum-II USB 2.0 chip, but I believe the issues is a USB/UVC issue and not a problem with the chip or what it is attempting to do. There is a slightly larger hive mind of UVC than the FTDI host chip. I have tried to find the magic incantation; I have tried yelling 4-letter words at the chip, and have even asked nicely, “Please” to no avail. The code was original derived from FTDI’s WebCam demo program which was very poorly written and contains a large number of un-described hardcoded numeric constants.

I have looked at every USB/UVC drivers (mostly Linux and Windows) on the entire internet. Calling all USB Host gurus for help please.

Thank you!


Title: Re: USB HOST UVC Camera FTDI Vinculum-II not reading video stream
Post by: FTDI Community on April 09, 2021, 05:24:15 PM
Hello,

It only works with the one particular camera model that could send data in such a low resolution to be suitable for both the VNC2 and the OLED display.

So, in this example code there is a deliberate dependency on the Logitech Webcam Pro 9000 camera for input. The output format, frames and resolution for the chosen camera and the OLED display are matched.

As for all UVC devices, the USB isochronous reads are matched exactly to the camera output agreed upon by the SET_CUR/GET_CUR during probe exchanges. The parameters requested in the SET_CUR probes are fixed (WebCamApp.c line 320) to what this camera supports and there is no verification in the code that this was accepted by the camera. If the chosen camera does not have a matching mode then it will respond with invalid values in the VProbeAndCom structure in the GET_CUR reply to the probe (line 332) request. You can check the response to the probe request for your cameras at this point before the commit is sent (line 348). The GET_MIN and GET_MAX requests can be used to query likely modes that cameras can support.

The VNC2 is a very mature IC and only operated at Full Speed USB which brings limitations.

Our sister company Bridgetek have an MCU with High Speed USB which is more suited to UVC cameras:

https://brtchip.com/ft900/ (https://brtchip.com/ft900/)

The FT9xx Modules (https://brtchip.com/m-ft9xx/) (eg MM900EV3A) include an OmniVision OV9655 camera module (1 megapixel).
This uses raw images and doesn’t support compressed images which limits throughput.

FT90x UVC Webcam (https://brtchip.com/SoftwareExamples-ft90x/#FT90x UVC Webcam) example includes support for an OmniVision OV5640 (5 megapixel) which is on the CleO Camera (http://brtchip.com/m-cleo/) module.
This supports compressed MPEG images (compressed) which allows for higher throughputs.
Note some hardware modifications are required to support this camera:

TPS76915DBVR (1.5V) for U17.
FT531GA (2.8V) for U16 and remove C109.

Also there are known issues with the FT90x UVC Webcam example code on our website. We are in the process of uploading a new working version (which can be downloaded on our FTP server for now: AN_414_FT90x_USBD_UVC_Webcam rev 2131.zip (http://ftp://u45902898-ide:Ftd1$erv@ftp.ftdichip.com/CES/FT9xx/AN_414_FT90x_USBD_UVC_Webcam rev 2131.zip)).
The example code supports the following resolutions:

QVGA: 320 x 240
VGA: 640x480
SVGA: 800 x 600

Uncompressed at 30fps would only work for QVGA. Lowering the fps would allow for higher resolution.
Compressed at 30fps would work at SVGA.

Higher resolutions and frame rates can be achieved with a different senor and settings.

The FT9xx Toolchain (http://brtchip.com/ft9xx-toolchain/) can be used to build, program and make changes to the example code.

Also please note that the MM900EV3A contains RevB silicon. The latest is RevC.
We only have MM900EV1B which contains RevC silicon but we don’t have any RevC hardware with a camera.
The RevB should be ok for test and evaluation purposes though.
You can refer to the following documents:

• PCN BRT 005  (https://brtchip.com/wp-content/uploads/Support/Documentation/PCNs/PCN_BRT_005.pdf)
• BRT AN 019 Migration Guide Moving from FT90x Revision B to FT90x Revision C (https://brtchip.com/wp-content/uploads/Support/Documentation/Application_Notes/ICs/MCU/BRT_AN_019_Migration_Guide_Moving_from_FT90x_Revision_B_to_FT90x_Revision_C.pdf)

Best Regards,
FTDI Community
Title: Re: USB HOST UVC Camera FTDI Vinculum-II not reading video stream
Post by: lzerman on April 10, 2021, 07:31:32 AM
Thank you for replying as well of the repeated history of the very limited scope of the WebCam demo code. I am fully familiar with every single line of that code, the SET_CUR/GET_CUR and in code can printout all of the valid values in every USB structure and the VideoProbeAndCommiteControl_t VProbeAndCom structures. I can post all of the camera USB dumps, all of the USB ioctl structure dumps, etc, but held off overwhelming the question with that much detail. That is why I took the time to create a special one-off program to demonstrate the problem in literally the shortest amount of code possible.

>>
… the USB Host (FTDI Vinculum-II) connects to a UVC camera, properly enumerates the device interfaces, end points, etc
… The Host -> UVC camera ioctl r/w work fine and all values … are validated properly against the USB dumps
<<

There are so many details of the larger program that I did not believe were necessary to cover in my initial question.

The “real” app will be reading a USB 2.0 720p@30fps when it is in MJPEG mode. I have already profiled the camera stream on Windows and Mac. On average each (and almost all) JPEG frames (in the mjpeg) runs less than 45k-bytes per frame. 30 FPS * 45k-bytes * 8-bits = 10,800,000bits/sec (10m-bit/sec) well within the 12m-bits/sec of Full Speed. If a frame gets dropped it is ok. The only thing on the USB bus is the camera since this is an embedded application. No bus contention with any other device. I considered doing BULK transfer, but the non-error recovery of ISO is (more) appropriate in this case.

Because of NDA I cannot disclose why we must use a USB camera vs the OmniVision cameras. The project started with the medical grade very tiny Omnivision OCHSA10 camera. I have been dealing directly with the medical group of Omnivision and in the end they cannot supply us what is required as they had (incorrectly) represented that they could. As well, if you are not selling at least 2-million cameras per year, (year-1 we may be 10k) you are not even a blip on their radar for their tiniest cameras.

So, can we please be moving beyond in this discussion of the hard-coded nature and limited scope of the WebCam sample and any potential help on the actual problem at hand. As I stated previously: “Bottom line; find a fix for this program and the same will work for the real program.” This is what I need help with pleeeeasssse!

I have 40-years of programming and EE hardware design experience, much of with embedded controllers. I know what I am doing with both USB and with UVC cameras, but once in a while in a career, you come across a real-bugger of an issue that can stump the best. I believe I am just missing a one-liner or even one-biter detail. That is where I am now and asking for assistance focused on the question at hand since this is what I am working on, and not a suggested change of chip to the FT90x which I am already familiar with but deemed not the right chip for this purpose.

Thank you,

Leonard


 
Title: Re: USB HOST UVC Camera FTDI Vinculum-II not reading video stream
Post by: FTDI Community on April 13, 2021, 04:52:50 PM
Hello,

We plugged your code into a VNC2 and a Logitech web cam. The code worked straight away with a transfer size of 192 bytes using the simple GET_CUR (probe) and SET_CUR (commit). We used a USB Analyser to observe the output. This particular webcam defaulted to a resolution 320x240 at 15 fps in MJPEG. There were 12 bytes of header per packet leaving up-to 180 bytes of data.

However the program printed out zero bytes per read. The cause is the USB Host read and writes work slightly differently from other read and writes. When a USB host read or write is used the actual length of data transfer is held in the usbhost_xfer_iso_t structure and not in the parameters passed to vos_dev_read/write.

See the page “USB Host Read and Write Operations” in the help file.
“In vos_dev_write() the num_to_write and num_written parameters are ignored and in vos_dev_read() the num_to_read and num_read parameters are ignored as this information is provided and returned in the transfer block.”

Specifically for isochronous transfers the maximum transfer length is specified in the len structure member of the usbhost_xfer_iso_t structure. In the “USB Host Isochronous Transfer Block” page the note for this member says “When the operation completes, the value of len is not updated. However, the transfer length for each frame transaction is updated.” So, the total count of bytes transferred is obtained by reading and checking the condition code of all the deployed len_psw members and summing the sizes there. This is done to ensure that any missed isochronous transfers are accounted for properly in code.

We’ve modified the code as follows and can see 192 bytes reported in the while loop correctly.

                xfer.frame = frame + 1;
               
                *num_read = 0; // LAZ, Added  /*Note this is a pointer and needs dereferenced. */
               
                // now start the read from the ISO endpoint (line 511)
                status = vos_dev_read(ctx->hc, (unsigned char *) &xfer, sizeof(usbhost_xfer_t), num_read); /*(2)*/

                // Sum the total data transferred in each of the isochronous packets for the actual transfer size.
                while (i > 0)
                {
                                *num_read += xfer.len_psw[i - 1].size;
                                i--;
                }
               
                if (status != USBHOST_OK)
                {
                                return UVC_ERROR;
                }

If we set the number of transfers in the usbhost_xfer_iso_t structure to 8 via the ISO_TRANSFER_COUNT macro and disable the printf then we can get meaningful measurements of data transfer from the web cam. In this default resolution the bus utilisation was between 10% and 25%. Maximum throughput can be achieved with the USBHOST_XFER_FLAG_NONBLOCKING set and semaphores used to signal completion as in the original app note code.

Best Regards,
FTDI Community
Title: Re: USB HOST UVC Camera FTDI Vinculum-II not reading video stream
Post by: lzerman on April 21, 2021, 08:59:35 AM
Thank you so much for addressing the problem directly at hand. Sadly, I am still not successfully reading bytes yet. Between the time I posted, and I received your response I had already corrected the issues that you pointed out.  I think my problem may lay with the configuration of the camera. I still get 0 bytes read upon return:

xfer.len_psw[i - 1].size == 0
xfer->len_psw[I - 1].cond_code == 9 (SBHOST_CC_DATAUNDERRUN)
Which makes sense if it is not reading the proper amount.

I have tried multiple different Formats and Frames
FormatIndex 1 = Uncompressed
FormatIndex 2 = MJPEG

As well as different FrameIndex
FrameIndex = 1 (Default 640x480)
FrameIndex = 2 (160x120)
FrameIndex = 5 (320 x 240)

To reproduce the same MJPEG 320 x 240 @ 15fps I configure:
// Add lines between the GET_CUR and SET_CUR
VProbeAndCom.bFormatIndex = 2; // MJPEG
VProbeAndCom.bFrameIndex = 5; // 320 x 240
VProbeAndCom.dwFrameInterval = 0x000A2C2A;  // 0x000A2C2A = 66.666600 mSec (15.00 Hz)

Still no bytes read. I have attached the dump of the Logitech camera I am using to see if you may find a flaw.

1.)   When you tested, did you actually do a debug dump of the buffer that the bytes are being read into or only just view the USB Analyzer?
2.)   What chip pin count where you using in the V2Eval? (I am using 48 pin.)

Thank you again for your help on the specifics of what I am doing.

Leonard
Title: Re: USB HOST UVC Camera FTDI Vinculum-II not reading video stream
Post by: FTDI Community on April 22, 2021, 05:22:00 PM
Hello,

We just read the number of bytes in the buffer. We did not output or examine the buffer contents. We could see the values were correct on the USB analyser.

The USBview output is taken when the device is connected to a High-Speed USB host. There will be other speed descriptor for Full-speed USB hosts.

Can this be plugged into a Full-speed host for USB? If not then the USBDescritptor sample code will dump the Full-speed descriptors to the UART. The descriptor may be different for Full-speed.

Which endpoint is being used? Is it being correctly selected by the right Alternate Setting?

Best Regards,
FTDI Community