I remember seeing once the following photo from Flickr, and having my brain melt slightly from trying to figure out what went wrong:
The issue was the propeller was rotating as the camera detector ‘read out’, i.e. there was some motion during the exposure of the camera. This is an interesting thing to think about, lets have a look.
Many modern digital cameras use as their ‘sensing’ device a CMOS detector, also known as an active-pixel sensor, which works by accumulating electronic charge as light falls upon it. After a given amount of time, the exposure time, the charge is shifted row-by-row back to the camera for further processing. There is then a finite time where the camera scans down the image, saving rows of pixels at a time. If there is any motion over this timescale the image will be distorted.
To illustrate, consider photographing a spinning propeller. In the animations below the red line corresponds to the current readout position, and the propeller continues to spin as the readout proceeds. The portion below the red line is saved as the captured image.
First, a propeller which completes 1/10th of a rotation during the exposure:
Some distortion, but nothing crazy. Now a propeller moving 10 times quicker, which completes a full rotation during the exposure:
This is starting to look like the Flickr image at the beginning. 5 times per exposure:
This is a little too far, things have clearly gone mental. Just for fun, let’s see what some different objects look like at different rotation speeds, from 0 to 1 rotation per exposure.
The same propeller as above:
A fatter propeller:
A car tire:
We can think of the rolling shutter effect being some coordinate transformation from the ‘object space’ of the real-world object, to the ‘image space’ of the warped image. The animation below shows what happens to the Cartesian coordinate grid as the number of rotations is increased. For small rotations the deformation is slight, as the number increases to 1 each side of the grid is moved successively towards the right-hand side of the image. This is a fairly complicated transformation to look at, but simple to understand.
Let the image be denoted by , and the real object (which is rotating) be denoted by where are 2D polar coordinates. Polar coordinates are a natural choice for this problem due to the rotational motion of the objects.
The object is rotating at angular frequency , and the shutter progresses across the image at speed in the vertical direction. At position in the image, the distance the shutter has moved since the start of exposure is , and so the time elapsed is . In this time the object has rotated a number of radians . Putting this together,
which is the required transformation. The factor is proportional to the number of rotations during the exposure, and parameterises the transformation.
To get some insight into the apparent shapes of the propellers, we can consider an object consisting of propellers where is non-zero only for for . The image is then non-zero for
In Cartesian coordinates this becomes
which helps to explain why the propellers get that S-shaped look – it’s just an inverse tangent function in the image space. Cool. I’ve plotted this function below for a set of 5 propeller blades at slightly different initial offsets, as might be observed during a video recording. They look pretty much like the shapes in the animations above.
Now we understand a little more about the process, can we do anything about these ruined photos? Taking one of the warped images above, I can take a line through it, rotate backwards the appropriate amount, then stick those pixels onto a new image. In the animation below I scan through the image on the left, marked by the red line, then rotate the pixels along that line onto a new image. This way we can build a picture of what the real object looks like even if a pesky rolling shutter ruined our original image.
Some kind commenters were good enough to extract the propeller blades for me. To figure out the real number of blades and the rotation velocity we can look to this excellent post at Daniel Walsh’s Tumblr blog here, where he definitely has the edge on mathematical explanation. He works out that we can calculate the number of blades by subtracting the ‘lower’ blades from the ‘upper’ blades, so in this picture we know there should be 3. We also know the propeller is rotating approximately 2 times during the exposure, so if we try ‘undoing’ the rotation with a few different speeds around that we get something like this:
I’ve had to guess where the centre of the propeller is, and I’ve drawn a circle to guide the eye. Looking at that, the centre shouldn’t be too far off. There is unfortunately a missing blade, but there’s still enough information to make an image.
There is a sweet spot where everything overlaps the most, so picking this rotation speed (2.39 rotations per exposure), the original image and blades look like this:
It’s still a bit of a mess unfortunately, but at least looks something like the real object. Thanks commenters, and the comment on Hacker News where I saw the link to Daniel’s post.