What is a camera?

Posted Oct 4, 2024 15:49 UTC (Fri) by Sesse (subscriber, #53779)
In reply to: What is a camera? by ribalda
Parent article: Coping with complex cameras

> All that said. v4l is a beautiful piece of engineering for the "capture part".

Not sure if I agree; as a userspace programmer, I just gave up supporting V4L2 input at some point because the API was so painful and bare-bones. Every single camera under the sun seems to support a slightly different set of formats and settings, and the onus is on you as an application to figure out which ones to support (e.g. you'll frequently need to embed a JPEG decoder!). At some point, I wanted to support _output_ via v4l2loopback, but that means you'll need to go through the format dance again; browsers and other clients will accept only a set of formats and nothing will try to convert for you. Eventually I went to the point of looking at the Chromium source and picking the format it accepted that was the least pain for me to create. :-) Thankfully, I only needed 720p60, so I didn't have to worry about fast DMA between my GPU and V4L2 buffers.

to post comments

What is a camera?

Posted Oct 8, 2024 14:44 UTC (Tue) by laurent.pinchart (subscriber, #71290) [Link] (4 responses)

> > All that said. v4l is a beautiful piece of engineering for the "capture part".
>
> Not sure if I agree;

I concur. We've made lots of mistakes in V4L2 over the years, and I'm responsible for some of them. That's not specific to V4L2 though, the whole kernel is developed by making mistakes and then trying to fix them. The important part is to constantly improve the APIs and implementation. And make new mistakes along the way to replace the old ones :)

This being said, I think that V4L2 is in a much better place than it was 10 years ago for cameras and ISPs. Don't use the API directly in your applications though. While V4L2 was designed as an application API, the world has moved on and we now need and increasingly have userspace frameworks to handle the hardware complexity.

What is a camera?

Posted Oct 8, 2024 15:49 UTC (Tue) by Sesse (subscriber, #53779) [Link] (3 responses)

Do you have any good recommendations—is it libcamera that one would want to use? (I'd rather not go to the level of FFmpeg or Gstreamer if I can avoid it; FFmpeg is a great codec library but pretty weak on anything involving live, and Gstreamer is just a world of pain)

What is a camera?

Posted Oct 8, 2024 18:34 UTC (Tue) by laurent.pinchart (subscriber, #71290) [Link] (2 responses)

> Do you have any good recommendations—is it libcamera that one would want to use? (I'd rather not go to the level of FFmpeg or Gstreamer if I can avoid it; FFmpeg is a great codec library but pretty weak on anything involving live, and Gstreamer is just a world of pain)

It depends on your use cases. For desktop applications, the future is Pipewire, which itself will interface with libcamera (or for the time being directly with V4L2 for USB webcams). For more specific applications, especially in embedded and IoT use cases, I recommend GStreamer with the libcamerasrc element. On Android one would of course use the libcamera Android adaptation layer that implements the camera HAL3 API. Direct usage of the libcamera API isn't something I expect to see very commonly for generic-purpose applications.

What is a camera?

Posted Oct 8, 2024 18:45 UTC (Tue) by Sesse (subscriber, #53779) [Link] (1 responses)

FWIW, my application is low-level enough to already talk to ALSA directly (both for PCM and for MIDI) and have its own userspace USB drivers for certain capture cards, so it sounds like libcamera directly would be the most likely avenue to try first. :-)

What is a camera?

Posted Oct 8, 2024 21:16 UTC (Tue) by laurent.pinchart (subscriber, #71290) [Link]

If you're dealing with systems that require controlling an ISP, libcamera would then be a good way forward. If you're dealing with UVC webcams, libcamera could still help but is probably overkill as it won't give you the features you mentioned were missing (such as transcoding between formats or decoding JPEG). If you're dealing with IP cameras, libcamera will definitely not help as we don't support those.