仕事上の事情から、いまUVCカメラ（USB接続のいわゆるWebカメラ）の仕様と制御方法について研究している。その一環として、libusbとlibuvcというライブラリにフォーカスして調査を行っている。
普段使っているMacでlibuvcをビルドして動かすことをやってみた。
libuvcのビルド
プラットフォームはmacOS Ventura、ビルド環境はHomebrewを使っている。
最初にlibuvcのビルドに必要な依存パッケージをインストールした。
% brew install cmake pkg-config % brew install libusb jpeg-turbo
実際の操作順は逆で、最初にlibuvcのソースを取得して、それをビルドしながら上のパッケージが必要なことが判ったのだが。
続いて、libuvcのソースを取得した。
% git clone https://github.com/libuvc/libuvc.git
そして、ビルドを行う。
% cd libuvc % mkdir build % cd build % cmake .. -- libusb-1.0 found using pkgconfig -- Found JPEG: /usr/local/lib/libjpeg.dylib (found version "80") -- Found JPEG library using standard module CMake Warning (dev) at /usr/local/Cellar/cmake/3.26.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:438 (message): The package name passed to `find_package_handle_standard_args` (JPEG) does not match the name of the calling package (JpegPkg). This can lead to problems in calling code that expects `find_package` result variables (e.g., `_FOUND`) to follow a certain pattern. Call Stack (most recent call first): cmake/FindJpegPkg.cmake:58 (find_package_handle_standard_args) CMakeLists.txt:45 (find_package) This warning is for project developers. Use -Wno-dev to suppress it. -- Found JPEG: /usr/local/lib/libjpeg.dylib -- Building libuvc with JPEG support. -- Configuring done (0.7s) -- Generating done (0.1s) -- Build files have been written to: /Users/LOGIN_USER/Builds/libuvc/build % make [ 4%] Building C object CMakeFiles/uvc.dir/src/ctrl.c.o [ 9%] Building C object CMakeFiles/uvc.dir/src/ctrl-gen.c.o [ 13%] Building C object CMakeFiles/uvc.dir/src/device.c.o [ 18%] Building C object CMakeFiles/uvc.dir/src/diag.c.o [ 22%] Building C object CMakeFiles/uvc.dir/src/frame.c.o [ 27%] Building C object CMakeFiles/uvc.dir/src/init.c.o [ 31%] Building C object CMakeFiles/uvc.dir/src/stream.c.o [ 36%] Building C object CMakeFiles/uvc.dir/src/misc.c.o [ 40%] Building C object CMakeFiles/uvc.dir/src/frame-mjpeg.c.o [ 45%] Linking C shared library libuvc.dylib [ 45%] Built target uvc [ 50%] Building C object CMakeFiles/uvc_static.dir/src/ctrl.c.o [ 54%] Building C object CMakeFiles/uvc_static.dir/src/ctrl-gen.c.o [ 59%] Building C object CMakeFiles/uvc_static.dir/src/device.c.o [ 63%] Building C object CMakeFiles/uvc_static.dir/src/diag.c.o [ 68%] Building C object CMakeFiles/uvc_static.dir/src/frame.c.o [ 72%] Building C object CMakeFiles/uvc_static.dir/src/init.c.o [ 77%] Building C object CMakeFiles/uvc_static.dir/src/stream.c.o [ 81%] Building C object CMakeFiles/uvc_static.dir/src/misc.c.o [ 86%] Building C object CMakeFiles/uvc_static.dir/src/frame-mjpeg.c.o [ 90%] Linking C static library libuvc.a [ 90%] Built target uvc_static [ 95%] Building C object CMakeFiles/example.dir/src/example.c.o [100%] Linking C executable example [100%] Built target example
libuvcの動作確認
ビルドしたlibuvcの動作確認として、UVCカメラを接続した状態で以下のコマンドを実行してみた。
% ./example UVC initialized Device found uvc_open: Access denied (-3) UVC exited
どうもrootユーザでやらないとダメみたいなので、sudo付きで実行してみたら下のような出力になって、これで動いているようだ。
% sudo ./example
Password:
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (1d6c:0103/AN20200825001) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 131
Formats:
MJPEGFormat(1)
bits per pixel: 0
GUID: 4d4a5047000000000000000000000000 (MJPG)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 4147200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(2)
capabilities: 00
size: 2560x1440
bit rate: 18432000-55296000
max frame size: 7372800
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(3)
capabilities: 00
size: 320x240
bit rate: 18432000-55296000
max frame size: 153600
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(4)
capabilities: 00
size: 640x360
bit rate: 18432000-55296000
max frame size: 460800
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(5)
capabilities: 00
size: 800x600
bit rate: 18432000-55296000
max frame size: 960000
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(6)
capabilities: 00
size: 960x720
bit rate: 18432000-55296000
max frame size: 1382400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(7)
capabilities: 00
size: 1024x576
bit rate: 18432000-55296000
max frame size: 1179648
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(8)
capabilities: 00
size: 640x480
bit rate: 18432000-55296000
max frame size: 614400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(9)
capabilities: 00
size: 1280x720
bit rate: 18432000-55296000
max frame size: 1843200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(10)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 4147200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(11)
capabilities: 00
size: 2560x1440
bit rate: 18432000-55296000
max frame size: 7372800
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(12)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 4147200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameFormat(2)
bits per pixel: 0
GUID: 4832363400001000800000aa00389b71 (H264)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(2)
capabilities: 00
size: 2560x1440
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(3)
capabilities: 00
size: 320x240
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(4)
capabilities: 00
size: 640x360
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(5)
capabilities: 00
size: 800x600
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(6)
capabilities: 00
size: 960x720
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(7)
capabilities: 00
size: 1024x576
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(8)
capabilities: 00
size: 640x480
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(9)
capabilities: 00
size: 1280x720
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(10)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(11)
capabilities: 00
size: 2560x1440
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(12)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
UncompressedFormat(3)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 640x480
bit rate: 18432000-55296000
max frame size: 614400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(2)
capabilities: 00
size: 1280x720
bit rate: 18432000-55296000
max frame size: 1843200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
END DEVICE CONFIGURATION
First format: (MJPG) 1920x1080 30fps
bmHint: 0001
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 333333
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 0
dwMaxVideoFrameSize: 4147200
dwMaxPayloadTransferSize: 3060
bInterfaceNumber: 1
Streaming...
Enabling auto exposure ...
... enabled auto exposure
callback! frame_format = 7, width = 1920, height = 1080, length = 197981, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 219087, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 242720, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 263191, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 293221, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 341212, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 356177, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 352805, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 350672, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 350672, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 331834, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 330931, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 330550, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 329433, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 329433, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 308818, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 309529, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 310410, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 311284, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 311284, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 310782, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 310160, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 310327, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 309760, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 310405, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 310493, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 311158, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 310334, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 309190, ptr = 0x3039
callback! frame_format = 7, width = 1920, height = 1080, length = 308535, ptr = 0x3039
exampleはsrc/example.cというソースからビルドされる実行ファイルだが、src/test.cというサンプルプログラム・ソースもあって、こちらはOpenCVを利用してカメラ映像の表示を行っている。
以下のコマンドを実行すると、このsrc/test.cも一緒にビルドできる。
% brew install opencv@3 % cmake .. -DBUILD_TEST=true -DOpenCV_DIR=/usr/local/opt/opencv@3/share/OpenCV % make
uvc_testというのがsrc/test.cからビルドされた実行ファイルだが、これを起動すると以下のようなメッセージが出力されてプログラムが停止してしまう。
% sudo ./uvc_test
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (1d6c:0103/AN20200825001) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 131
Formats:
MJPEGFormat(1)
bits per pixel: 0
GUID: 4d4a5047000000000000000000000000 (MJPG)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 4147200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(2)
capabilities: 00
size: 2560x1440
bit rate: 18432000-55296000
max frame size: 7372800
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(3)
capabilities: 00
size: 320x240
bit rate: 18432000-55296000
max frame size: 153600
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(4)
capabilities: 00
size: 640x360
bit rate: 18432000-55296000
max frame size: 460800
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(5)
capabilities: 00
size: 800x600
bit rate: 18432000-55296000
max frame size: 960000
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(6)
capabilities: 00
size: 960x720
bit rate: 18432000-55296000
max frame size: 1382400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(7)
capabilities: 00
size: 1024x576
bit rate: 18432000-55296000
max frame size: 1179648
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(8)
capabilities: 00
size: 640x480
bit rate: 18432000-55296000
max frame size: 614400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(9)
capabilities: 00
size: 1280x720
bit rate: 18432000-55296000
max frame size: 1843200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(10)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 4147200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(11)
capabilities: 00
size: 2560x1440
bit rate: 18432000-55296000
max frame size: 7372800
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(12)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 4147200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameFormat(2)
bits per pixel: 0
GUID: 4832363400001000800000aa00389b71 (H264)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(2)
capabilities: 00
size: 2560x1440
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(3)
capabilities: 00
size: 320x240
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(4)
capabilities: 00
size: 640x360
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(5)
capabilities: 00
size: 800x600
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(6)
capabilities: 00
size: 960x720
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(7)
capabilities: 00
size: 1024x576
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(8)
capabilities: 00
size: 640x480
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(9)
capabilities: 00
size: 1280x720
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(10)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(11)
capabilities: 00
size: 2560x1440
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(12)
capabilities: 00
size: 1920x1080
bit rate: 18432000-55296000
max frame size: 0
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
UncompressedFormat(3)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 640x480
bit rate: 18432000-55296000
max frame size: 614400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
FrameDescriptor(2)
capabilities: 00
size: 1280x720
bit rate: 18432000-55296000
max frame size: 1843200
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
END DEVICE CONFIGURATION
bmHint: 0001
bFormatIndex: 3
bFrameIndex: 1
dwFrameInterval: 333333
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 0
dwMaxVideoFrameSize: 614400
dwMaxPayloadTransferSize: 3060
bInterfaceNumber: 1
Streaming for 10 seconds...
set_ae_mode: Pipe (-9)
set_exp_abs: Pipe (-9)
callback! length = 3048, ptr = 12345
2023-07-10 10:40:13.082 uvc_test[62190:4733991] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSWindow drag regions should only be invalidated on the Main Thread!'
*** First throw call stack:
(
0 CoreFoundation 0x00007ff816f7043b __exceptionPreprocess + 242
1 libobjc.A.dylib 0x00007ff816abfe25 objc_exception_throw + 48
2 CoreFoundation 0x00007ff816f985d6 _CFBundleGetValueForInfoKey + 0
3 AppKit 0x00007ff81a03f161 -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 307
4 AppKit 0x00007ff81a02bf4f -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1247
5 AppKit 0x00007ff81a02ba69 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 42
6 AppKit 0x00007ff81a2f9ea2 -[NSWindow initWithContentRect:styleMask:backing:defer:screen:] + 50
7 libopencv_highgui.3.4.16.dylib 0x0000000107384261 cvNamedWindow + 344
8 uvc_test 0x00000001072b7ac0 cb + 192
9 libuvc.0.0.7.dylib 0x000000010736b59f _uvc_user_caller + 127
10 libsystem_pthread.dylib 0x00007ff816e1d259 _pthread_start + 125
11 libsystem_pthread.dylib 0x00007ff816e18c7b thread_start + 15
)
libc++abi: terminating with uncaught exception of type NSException
zsh: abort sudo ./uvc_test
この例外エラーは、メインスレッドの外からUIを叩こうとすると起きるものらしい。
src/test.cのソース内に以下のような記述があって、cvShowImageというOpenCVの関数を呼び出したときに起きているようだ。
void cb(uvc_frame_t *frame, void *ptr) { .... .... printf("callback! length = %u, ptr = %d\n", frame->data_bytes, (int) ptr); .... .... .... .... cvImg = cvCreateImageHeader( cvSize(bgr->width, bgr->height), IPL_DEPTH_8U, 3); cvSetData(cvImg, bgr->data, bgr->width * 3); cvNamedWindow("Test", CV_WINDOW_AUTOSIZE); cvShowImage("Test", cvImg); cvWaitKey(10); cvReleaseImageHeader(&cvImg); uvc_free_frame(bgr); } int main(int argc, char **argv) { .... .... .... .... res = uvc_init(&ctx, NULL); if (res < 0) { uvc_perror(res, "uvc_init"); return res; } puts("UVC initialized"); .... .... .... .... .... .... .... .... res = uvc_start_streaming(devh, &ctrl, cb, 12345, 0); if (res < 0) { uvc_perror(res, "start_streaming"); } else { puts("Streaming for 10 seconds..."); .... .... } sleep(10); uvc_stop_streaming(devh); puts("Done streaming."); } .... .... .... .... uvc_exit(ctx); puts("UVC exited"); return 0; }
Ubuntuでもlibuvcのビルドをやってみたが、Ubuntuではuvc_testは問題なく動作してカメラ映像が表示されることが確認できている。つまり、上記の現象はmacOS固有の障害ということになる。
SwiftだとDispatchQueue.main.syncという関数があって、これを使ってメインスレッド側でUI処理を行うようにすれば解決できるらしい。src/test.cはC言語記述でpthreadを使っているが、この場合、同じ方法をどうやって実現すれば良いのか判らなかった。
じつは、上記と同じ内容の作業を大分前にHigh Sierra上で実施済みだったのだが、今回MacのOSをVenturaにアップグレードしたので、それで本作業をもう一度やりなおして記事にした。
ちなみに、この研究の目的は、UVCカメラの映像取得・制御をネットワーク経由でできる環境を作ることだ。このような環境をLinux、Mac、Windowsのすべてで（欲を言えば、AndroidとiOSでも）実現することを最終目標として取り組んでいる。
