Raspberry Pi Camera Sensor Details
The reason I wrote this up was to find out what is possible now and into the future using the Raspberry Pi Camera. There were features that were not included when the RasPi Camera was originally released and have been added since. Since you can not do things the sensor can not I wanted to learn everything about the sensor to know what else might be added and what is just not possible.
A few things came to light that I was unaware of.
One of them was that the GPU handles most of what I call the heavy lifting and not the sensor.
2 other things that peaked my interest were Flash and Stereo.
Flash trigger is not currently implemented because of the Automatic Gain Control and timing necessary to implement that.
The second is Stereo Cameras, which are totally possible. Since the RasPi Model A and B only have one Camera CSI Interface that is not even an option for them. The Compute module is capable of Stereo cameras but that functionality IS NOT implimented so don't run out and buy one as IT WON'T WORK.
I would love to daisy chain the cameras between Raspi's to Sync them all at the sensor level, also NOT CURRENTLY POSSIBLE AND PROBABLY NEVER WILL BE JUST A PERSONAL DREAM OF MINE.
Now off to what I found out
Here is the forum post where I asked the questions about the sensor, Forum Post
Raspberry Pi Sensor
Sensor Type - OmniVision OV5647 - Webpage
This sensor uses what is called a 'Rolling Shutter' Wikipedia Description of Rolling Shutter
1.4 µm by 1.4 µm pixel with OmniBSI technology for high performance (high sensitivity, low crosstalk, low noise)
Optical size of 1/4"
Automatic image control functions:
Automatic exposure control (AEC)
Automatic white balance (AWB)
Automatic band filter (ABF)
Automatic 50/60 Hz luminance detection
Automatic black level calibration (ABLC)
Note on Image Control - All N/A as far as the sensor is concerned. Done by GPU. There are manual controls for AEC and AWB. 50/60Hz flicker avoidance is also supported.
Programmable controls for:
Frame rate - Variable
2592 by 1944 1-15fps, video or stills mode, Full sensor full FOV, default stills capture
1920 by 1080 1-30fps, video mode, 1080p30 cropped
1296 by 972 1-42fps, video mode, 4:3 aspect binned full FOV. Used for stills preview in raspistill.
1296 by 730 1-49fps, video mode, 16:9 aspect , binned, full FOV (width), used for 720p
640 by 480 42.1-60fps, video mode, up to VGAp60 binned
640 by 480 60.1-90fps, video mode, up to VGAp90 binned
AEC/AGC 16-zone size/position/weight control - Done by the GPU, and that can also set up AGC regions
Mirror and flip - Done by Sensor
Cropping - Apart from the 1080P mode, there is no cropping done on the sensor, it is all done by the GPU
Windowing - From Datasheet - "An image windowing area is defined by four parameters... By properly setting the parameters, any portion or size within the sensor array can be defined as an visible area. This windowing is achieved by simply masking the pixels outside the defined window; thus it will not affect the original timing.
Panning - My guess is that this has to do moving the above Windowing
Image quality controls:
Lens correction - The lens shading correction is currently 'hardcoded'. An automatic lens shading algorithm exists and may be added in the future
Defective pixel canceling - Done by GPU
Support for output formats:
8-/10-bit RAW RGB data - Bayer RAW10 data directly from sensor Conversion tool to DNG - All other formats are handled by the GPU
Support for video or snapshot operations - Yes you can capture stills or video
Support for LED and flash strobe mode - Not done as yet. Could only support an LED flash as xenon really needs a global reset and shutter. Flash needs integrating with AGC control to ensure correct exposure. Some flash drivers allow configurable intensity of the LED, so you end up needing a preflash to drive those calculations.
Support for internal and external frame synchronization for frame exposure mode - Frame sync would allow you to slave a second sensor to be exactly in sync for doing stereoscopic. The Model A and B only have a single camera interface exposed (the Compute Module has two exposed) and the software doesn't support stereoscopic at present, so no real demand.
Support for 2x2 binning for better SNR in low light conditions - I think this is used when you use the Night Option but I am not sure. Anything output res below 1296x976 will use the 2x2 binned mode
Post binning resampling filter to minimize spatial/aliasing artifacts on 2x2 binned image - Yes Don't know if the sensor is doing that for Bayer output (suspect not), but the GPU could be (not sure if it is or not)
Support for horizontal and vertical sub-sampling - Binning is a form of subsampling. There is also the option for skipping, which is used for the VGA 60 and 90fps modes
Standard serial SCCB interface - Yes, just means it is controlled over I2C
Digital video port (DVP) parallel output interface - One form of output data bus. Not used on Pi.
MIPI interface (two lanes) - This is the flat ribbon cable that connects the camera to the Raspberry Pi
32 bytes of embedded one-time programmable (OTP) memory - OTP - serial number, calibration data for AWB and/or lens shading correction. Any number of things.
On-chip phase lock loop (PLL) - Sensor clock runs at something silly. External clock fed to it is 25MHz IIRC. PLL converts between the two.
Embedded 1.5V regulator for core power - External power is 3.3V IIRC
Programmable I/O drive capability, I/O tri-state configurability - Marketing waffle. Lines tristate on powerdown probably
Support for black sun cancellation - Nice Explanation
One amazing feature that is an engineering marvel is Vectors from Coarse Motion Estimate - Link to Post Thanks Gordon
And Off to the Thanks on that note
Thanks Gert van Loo, JamesH, RasPi Forum 6by9, AndrewS