Thursday, March 06, 2008

How to Extract Video Still Frames with MPlayer

While working on a video content management system, I was in need of capturing frames from video files so they can be used as a preview for video. The system already had some code in place but it only worked for video encoding in Windows Media format (.wmv extension). That would not do, I thought, not in this day and age when we have so many file formats and video codecs. So I need to able to:

1. Capture video frames from within .NET code
2. Capture images from all kinds of different video formats
3. Not reinvent the wheel while satisfying #1 and #2

Enter MPlayer

Stolen directly from the Information page of MPlayer, "MPlayer is a movie player which runs on many systems (see the documentation). It plays most MPEG/VOB, AVI, Ogg/OGM, VIVO, ASF/WMA/WMV, QT/MOV/MP4, RealMedia, Matroska, NUT, NuppelVideo, FLI, YUV4MPEG, FILM, RoQ, PVA files, supported by many native, XAnim, and Win32 DLL codecs. You can watch VideoCD, SVCD, DVD, 3ivx, DivX 3/4/5 and even WMV movies..". To add my own description, MPlayer is an open source command line video player.

So What

Metallica has an old song called "So What", great tune for the metal heads in all of us. The answer is simple, MPlayer can do a lot more than play video. It can capture images, stream video over http and even transcode video from one format to another (but the last feature is grounds for another article).

You will need:

Setup

1. Extract the mPlayer executable (mplayer.exe)
2. Extract the mPlayer codecs in a directory called "codecs" in the same directory as the mPlayer executable
3. Add a reference to the mPlayerWrapper project or compiled DLL

Usage

• Capture with default arguments

in C#
// Specify the path to the video file
string videoFilePath = @"drive letter:\path\to\myVideo.mpg";
// Declare the mplayer instance with the mplayer executable residing in the
// same directory as your executable
mPlayerWrapper mPlayerInstance = new mPlayerWrapper();
// Capture frames
mPlayerInstance.captureFrames(videoFilePath);


in VB.NET
' Specify the path to the video file
dim videoFilePath As String = "drive letter:\path\to\myVideo.mpg"
' Declare the mplayer instance with the mplayer executable residing in the
' same directory as your executable
dim mPlayerInstance As new mPlayerWrapper()
' Capture frames
mPlayerInstance.captureFrames(videoFilePath)


This will capture 12 frames with 5 second interval between each frame and put them in the same directory as your video file. The filename of each frame will be "myVideo_thumb01.jpg" to "myVideo_thumb12.jpg". Each frame will be scaled to 270x200.
• Capture arguments
• mPlayerPath - sets the path where the mPlayer executable (mplayer.exe) is located. The default is in the same directory as your code is executing.
• currentFilePath - sets the file path to the video file you are using
• cleanOutputDirectory - deletes all the "jpg" images in the capture output directory before capturing
• captureInterval - sets the interval at which frames will be captured. Only applicable if using a time interval capture method (as outlined below)
• numberOfFramesToCapture - the number of frames to be captured
• captureExactNumberOfFrames - tells the wrapper to attempt to capture the exact number of frames as specified by the "numberOfFramesToCapture" property. This can be used if a file has too few frames but you still want to capture an exact number
• useTimeSeekToCapture - used to set the wrapper method of capture to seeking through the file instead of capturing a frame at an interval
• thumbnailPrefix - the prefix to be used when creating the filenames for captured frames. The default is the name of the video with "_thumb" append to it as in "myVideo_thumb01.jpg"
• capturedFrameWidthHeight - the width:height that each frame will be scaled to. The default is 270:200
• scaleCapturedFrames - used in conjunction with the "capturedFrameWidthHeight" property to scale down the captured frames. Set to true by default
• Capture in a different output directory
in C#
// Specify the path to the video file
string videoFilePath = @"drive letter:\path\to\myVideo.mpg";
// Specify the output directory
string outputPath = @"drive letter:\path\to\output directory";
// Declare the mplayer instance with the mplayer executable residing in the
// same directory as your executable
mPlayerWrapper mPlayerInstance = new mPlayerWrapper();
// Capture frames
mPlayerInstance.captureFrames(videoFilePath, outputPath);


in VB.NET
' Specify the path to the video file
dim videoFilePath As String = "drive letter:\path\to\myVideo.mpg"
' Specify the output directory
dim outputPath As String = @"drive letter:\path\to\output directory"
' Declare the mplayer instance with the mplayer executable residing in the
' same directory as your executable
dim mPlayerInstance As new mPlayerWrapper()
' Capture frames
mPlayerInstance.captureFrames(videoFilePath, outputDirectory)

• Using distinct capture methods:
1. captureFramesWithInterval - by default captures a frame every 5 seconds with up to 12 frames
2. captureFramesWithTimeSeek - by default captures 1 frame each second with up to 12 frames by seeking through the file
• Bonus
• getFileProperties - returns a SortedList of video and audio properties for the file
• getAudioProperties - returns a SortedList of audio properties for the file
• getVideoProperties - returns a SortedList of video properties for the file

mPlayerWrapper: http://blog.tech-cats.net/examples/dotnet/mPlayerWrapper.dll
mPlayerWrapper Source: http://blog.tech-cats.net/examples/dotnet/mplayerWrapper-v0.2.zip

1. What option could I use for capturing 1 frame every 200 frames?

2. You would need to use the time interval method but since you want 200 frames, first get the frames per second from the file info (getFileInfo function) and calculate the time interval for 200 frames.

3. This is perfect for a project I am doing but no matter what method I use I can't seem to capture faster than once every 30 seconds. I would like to capture every 5-10 seconds. Also I get an exception (Input string not in correct format) . . Any Thoughts?

4. Can you post the code that is trying to extract the frames? Either here or contact me over the chat widget and maybe I can help you.

5. I did everything as you explained.
The getVideoProperties method works fine, but the captureFrames*** methods create no files on disk (and produce no errors at the same time). What can be the reason?

Thank you,
Alex.

6. It could be your file or the lack of codecs. Email me (boyank at gmail) or talk to me on the chat here if you want me to help you.

7. Okay... So I created a new project and made the reference to the DLL file in the Bin\Debug folder of the mPlayerWrapper...

I use this code:

Imports System.IO

Public Class Form1

Private Sub Btn_Grab_Image_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Btn_Grab_Image.Click

Dim mPlayerInstance As New mPlayerWrapper
mPlayerInstance.captureFrames("C:\ImageGrabber\Video\Lake.wmv", "C:\ImageGrabber\Image\")

End Sub
End Class

I get this error message when I click the button:

The path is not of a legal form.

Any ideas?

8. I've updated the wrapper DLL and source code, that should fix the issues.

9. So I was curious to see if I could get mplayer to dump all frames from any segment.

With this wrapper it looks like its limited to int seconds. However, mplayer will accept something a little more granular:

mplayer -ss 00:00:00:01 -frames 72 -vo png input.mov

This will save the first 72 frames (at 24fps the first 3 seconds) as png from the begining.

It doesn't look like the wrapper supports this, but should be fairly trivial to add. Thought I'd pass on the info if anyone was interested.

10. Hi I've been working with this and seem to run into a strange bug - I'm trying to capture a frame every 60 seconds, but it seems to capture it every 62 seconds or so?? So by the end of a video, it loops around and for an hour video, the last two frames are from the start.
Have you run into this? Any known fix??

ex code:
mplayer "file.mp4" -nosound -really-quiet -vo "jpeg:outdir=directory" -vf "scale=320:240" -ss 0 -sstep 60 -frames 47

(For a 46:40 long video)
Thanks for the help

11. What you are experiencing is most likely due to the key frames in the file. The way the stepping works: first skips to the position and then captures the frame. However, if there is no key frame at the position, it doesn't actually go directly to that frame but to the next key frame. That's the best way I can explain the behavior you are seeing. Have you tried a different file?

12. Yes it is actually quite consistent with all of the files I have, it is almost always the same offset. Do you know a way around this so that the offset doesn't accumulate? It's strange that it's basically treating my "60" argument like "63" or so for the step size.
Thanks!

13. The only other suggestion I have is try the opposite method of capture. The wrapper has two methods: captureFramesWithInterval and captureFramesWithTimeSeek, so try the one you haven't used yet.

14. Can you give the time seek one multiple times, so give it a comma separated list of all frames every 1' or something? I tried that method before but a problem I had was that it would seem to lock up and get super slow if I tried to call mplayer for each frame separately.
Thanks!

15. Contact me off here at boyank at google or the chat, maybe I can help you if you haven't figured it out.

16. Hi all
I just found this wrapper and am wondering can I use this to get all the frames from a video file.

Thanks for any information.

17. You can but with some modifications to the command line parameters for mplayer.

18. Thanks for the information
I am wondering if this command would work
mplayer input.avi -vo jpeg:outdir=dirname

or would this be better
mplayer -ss 00:00:00:01 -vo jpeg input.avi

I will be working with an avi file

I found the first command on the mplayer wiki page

19. Yup, the first one works. However for a 37 seconds movie, I ended up with 949 images, a bit crazy.

20. Boyan - very helpful code. Thank you! It there a way mplayer.exe can take direct output path as a parameter for jpeg:outdir?

21. Vladimir, thanks! The mplayer.exe only takes a relative directory for the jpeg output. That's why (if you look at the source for the wrapper) if you want a different output directory, the images are first generated in a relative temp directory and then moved to the final destination.

22. I am having trouble finding the exact executables and codacs from the mPlayer website, it's pretty confusing. Can you zip everything needed together into a .zip file and post a download link, that way I will be sure to have the right files you created this tutorial using :)

Thanks a million, happy coding!

23. I just add dll to my project and copy tour code to the project.......but it produce nothing.........what should i do now please help me

24. This comment has been removed by a blog administrator.

// //]]>