I have been working with PlayStation 4 Camera since 2013. So i want to share with you all things learned since then.
First of all, if device has not a public/free driver , you need to make
reverse engineering and you need to look for information about its
chipset and make a research about it. So you need to open the device and
look for information about its different chips. After making some
research we can show the next high level design diagram:
PlayStation 4 Camera is using a propietary connector called AUX, however it is a USB 3 cable so our first goal was to cut off the AUX connector and connect it to an USB3 male connector, after this we can connect directly to Mac/PC with USB3 female ports. Next step was to get an USB sniffer capture from PlayStation 4 with a beagle device in the middle, without this we could not have learnt about how to initialize the device and their different options.
Chips description:
- OV580. It is an asic from Omnivision a typical custom USB bridge solution to manage all other chip. No information or datasheet about it without signing a NDA with Omnivision. Other companies using it are Mantis Vision and LeapMotion. I have worked with others Omnivision USB bridge solutions included in PlayStation EyeToy(OV519) and PlayStation Eye(OV534) so basically it is the same work i made with other drivers for Omnivision.
- OV9713. Two cmos sensors well documented with full datasheet available in google.
- AK5703. An analogical to digital converter connected with an microphone array (4). Datasheet is available. The datasheet was very helpfull to get audio properly from PlayStation 4 Camera.
- 4g51A. An eeprom chip. It contains the initial configuration for PlayStation 4 Camera.
When you connect device to a Mac/PC with USB3 it is in boot mode waiting for loading firmware. Analizing sniffer capture we got the firmware and steps needed to load it to device. So after loading firmware you have and special UVC Camera device and you can learn about their different streaming modes.
Interface 1 alt setting 0 describe all video modes with these possible options:
- mode 0 fps 60(default), 30, 15, 8 video left and video right frames 1280x800
- mode 1 fps 120(default), 60, 30, 15, 8 video left and video right frames 640x400
- mode 2 fps 240(default), 120, 60, 30 video left and video right frames 320x192
OV580 is delivering in each mode:
- mode 0 3448x2x808 bytes frame each row with:
header 32 bytes
audio 64 bytes
video left 1280x2 bytes
video right 1280x2 bytes
video interleave 840x2 bytes
- mode 1 1748x2x408 bytes frame each row with:
header 32 bytes
audio 64 bytes
video left 640x2 bytes
video right 640x2 bytes
video interleave 420x2 bytes
- mode 2 898x2x200 bytes frame each row with
header 32 bytes
audio 64 bytes
video left 320x2 bytes
video right 320x2 bytes
video interleave 210x2 bytes
Part of Header is showed in the next screenshot:
Part of Audio is showed in the next screenshot:
Audio has 4 channel i use Audicity tool loading a 10 second capture a reproducing 4 audio channels with success. AK5703 datasheet was very helpfull. An screenshot from audicity tool:
Videl Left and Right can be converted to RGB with CV_YUV2RGB_YUY2 conversion:
cv::Mat yuv(size_y,size_x,CV_8UC2 ,in);
cv::Mat rgb(size_y,size_x,CV_8UC3, out);
cv::cvtColor(yuv, rgb, CV_YUV2RGB_YUY2);
Video interleave it is also the same video format but it is making a mix of two sensor
A sample application made using basic PS4EYECam driver showing a 640x400 mode for two PlayStation Cameras at the same time:
Low level usb stuff is implemented and available in my repository. EyeLab is a poc of multi camera application using PS4EYECam driver and it is not in the repository, you don't need to make changes in the driver to support multi cameras you need to do it at application level.
So after this i can say that PlayStation 4 Camera is full documented and it can be supported on other platforms. All information needed is here and in my repository.
PlayStation 4 receives frames with format explained here. I suppose that secondary processor makes all the job to parse frames(video and audio) and connect with image/audio libraries from sdk(if someone want to share it with me you can contact with me :P). PlayStation 4 only has 1 AUX port so a multicam solution would be hard to implement like i am doing now in Osx.
Autor: Antonio Marquez aka bigboss