Search

Thursday, March 06, 2008

How to Extract Video Still Frames with MPlayer

While working on a video content management system, I was in need of capturing frames from video files so they can be used as a preview for video. The system already had some code in place but it only worked for video encoding in Windows Media format (.wmv extension). That would not do, I thought, not in this day and age when we have so many file formats and video codecs. So I need to able to:

  1. Capture video frames from within .NET code
  2. Capture images from all kinds of different video formats
  3. Not reinvent the wheel while satisfying #1 and #2

Enter MPlayer

Stolen directly from the Information page of MPlayer, "MPlayer is a movie player which runs on many systems (see the documentation). It plays most MPEG/VOB, AVI, Ogg/OGM, VIVO, ASF/WMA/WMV, QT/MOV/MP4, RealMedia, Matroska, NUT, NuppelVideo, FLI, YUV4MPEG, FILM, RoQ, PVA files, supported by many native, XAnim, and Win32 DLL codecs. You can watch VideoCD, SVCD, DVD, 3ivx, DivX 3/4/5 and even WMV movies..". To add my own description, MPlayer is an open source command line video player.

So What

Metallica has an old song called "So What", great tune for the metal heads in all of us. The answer is simple, MPlayer can do a lot more than play video. It can capture images, stream video over http and even transcode video from one format to another (but the last feature is grounds for another article).

Putting It Together

You will need:

Setup

  1. Extract the mPlayer executable (mplayer.exe)
  2. Extract the mPlayer codecs in a directory called "codecs" in the same directory as the mPlayer executable
  3. Add a reference to the mPlayerWrapper project or compiled DLL

Usage

  • Capture with default arguments

    in C#
    // Specify the path to the video file
    string videoFilePath = @"drive letter:\path\to\myVideo.mpg";
    // Declare the mplayer instance with the mplayer executable residing in the
    // same directory as your executable
    mPlayerWrapper mPlayerInstance = new mPlayerWrapper();
    // Capture frames
    mPlayerInstance.captureFrames(videoFilePath);
    

    in VB.NET
    ' Specify the path to the video file
    dim videoFilePath As String = "drive letter:\path\to\myVideo.mpg"
    ' Declare the mplayer instance with the mplayer executable residing in the
    ' same directory as your executable
    dim mPlayerInstance As new mPlayerWrapper()
    ' Capture frames
    mPlayerInstance.captureFrames(videoFilePath)
    

    This will capture 12 frames with 5 second interval between each frame and put them in the same directory as your video file. The filename of each frame will be "myVideo_thumb01.jpg" to "myVideo_thumb12.jpg". Each frame will be scaled to 270x200.
  • Capture arguments
    • mPlayerPath - sets the path where the mPlayer executable (mplayer.exe) is located. The default is in the same directory as your code is executing.
    • currentFilePath - sets the file path to the video file you are using
    • cleanOutputDirectory - deletes all the "jpg" images in the capture output directory before capturing
    • captureInterval - sets the interval at which frames will be captured. Only applicable if using a time interval capture method (as outlined below)
    • numberOfFramesToCapture - the number of frames to be captured
    • captureExactNumberOfFrames - tells the wrapper to attempt to capture the exact number of frames as specified by the "numberOfFramesToCapture" property. This can be used if a file has too few frames but you still want to capture an exact number
    • useTimeSeekToCapture - used to set the wrapper method of capture to seeking through the file instead of capturing a frame at an interval
    • thumbnailPrefix - the prefix to be used when creating the filenames for captured frames. The default is the name of the video with "_thumb" append to it as in "myVideo_thumb01.jpg"
    • capturedFrameWidthHeight - the width:height that each frame will be scaled to. The default is 270:200
    • scaleCapturedFrames - used in conjunction with the "capturedFrameWidthHeight" property to scale down the captured frames. Set to true by default
  • Capture in a different output directory
    in C#
    // Specify the path to the video file
    string videoFilePath = @"drive letter:\path\to\myVideo.mpg";
    // Specify the output directory
    string outputPath = @"drive letter:\path\to\output directory";
    // Declare the mplayer instance with the mplayer executable residing in the
    // same directory as your executable
    mPlayerWrapper mPlayerInstance = new mPlayerWrapper();
    // Capture frames
    mPlayerInstance.captureFrames(videoFilePath, outputPath);
    

    in VB.NET
    ' Specify the path to the video file
    dim videoFilePath As String = "drive letter:\path\to\myVideo.mpg"
    ' Specify the output directory
    dim outputPath As String = @"drive letter:\path\to\output directory"
    ' Declare the mplayer instance with the mplayer executable residing in the
    ' same directory as your executable
    dim mPlayerInstance As new mPlayerWrapper()
    ' Capture frames
    mPlayerInstance.captureFrames(videoFilePath, outputDirectory)
    
  • Using distinct capture methods:
    1. captureFramesWithInterval - by default captures a frame every 5 seconds with up to 12 frames
    2. captureFramesWithTimeSeek - by default captures 1 frame each second with up to 12 frames by seeking through the file
  • Bonus
    • getFileProperties - returns a SortedList of video and audio properties for the file
    • getAudioProperties - returns a SortedList of audio properties for the file
    • getVideoProperties - returns a SortedList of video properties for the file

Downloads

mPlayerWrapper: http://blog.tech-cats.net/examples/dotnet/mPlayerWrapper.dll
mPlayerWrapper Source: http://blog.tech-cats.net/examples/dotnet/mplayerWrapper-v0.2.zip

24 comments:

  1. What option could I use for capturing 1 frame every 200 frames?

    ReplyDelete
  2. You would need to use the time interval method but since you want 200 frames, first get the frames per second from the file info (getFileInfo function) and calculate the time interval for 200 frames.

    ReplyDelete
  3. This is perfect for a project I am doing but no matter what method I use I can't seem to capture faster than once every 30 seconds. I would like to capture every 5-10 seconds. Also I get an exception (Input string not in correct format) . . Any Thoughts?

    ReplyDelete
  4. Can you post the code that is trying to extract the frames? Either here or contact me over the chat widget and maybe I can help you.

    ReplyDelete
  5. I did everything as you explained.
    The getVideoProperties method works fine, but the captureFrames*** methods create no files on disk (and produce no errors at the same time). What can be the reason?

    Thank you,
    Alex.

    ReplyDelete
  6. It could be your file or the lack of codecs. Email me (boyank at gmail) or talk to me on the chat here if you want me to help you.

    ReplyDelete
  7. Okay... So I created a new project and made the reference to the DLL file in the Bin\Debug folder of the mPlayerWrapper...

    I use this code:

    Imports System.IO

    Public Class Form1

    Private Sub Btn_Grab_Image_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Btn_Grab_Image.Click

    Dim mPlayerInstance As New mPlayerWrapper
    mPlayerInstance.captureFrames("C:\ImageGrabber\Video\Lake.wmv", "C:\ImageGrabber\Image\")

    End Sub
    End Class

    I get this error message when I click the button:

    The path is not of a legal form.

    Any ideas?

    ReplyDelete
  8. I've updated the wrapper DLL and source code, that should fix the issues.

    ReplyDelete
  9. So I was curious to see if I could get mplayer to dump all frames from any segment.

    With this wrapper it looks like its limited to int seconds. However, mplayer will accept something a little more granular:

    mplayer -ss 00:00:00:01 -frames 72 -vo png input.mov

    This will save the first 72 frames (at 24fps the first 3 seconds) as png from the begining.

    It doesn't look like the wrapper supports this, but should be fairly trivial to add. Thought I'd pass on the info if anyone was interested.

    ReplyDelete
  10. Hi I've been working with this and seem to run into a strange bug - I'm trying to capture a frame every 60 seconds, but it seems to capture it every 62 seconds or so?? So by the end of a video, it loops around and for an hour video, the last two frames are from the start.
    Have you run into this? Any known fix??

    ex code:
    mplayer "file.mp4" -nosound -really-quiet -vo "jpeg:outdir=directory" -vf "scale=320:240" -ss 0 -sstep 60 -frames 47

    (For a 46:40 long video)
    Thanks for the help

    ReplyDelete
  11. What you are experiencing is most likely due to the key frames in the file. The way the stepping works: first skips to the position and then captures the frame. However, if there is no key frame at the position, it doesn't actually go directly to that frame but to the next key frame. That's the best way I can explain the behavior you are seeing. Have you tried a different file?

    ReplyDelete
  12. Yes it is actually quite consistent with all of the files I have, it is almost always the same offset. Do you know a way around this so that the offset doesn't accumulate? It's strange that it's basically treating my "60" argument like "63" or so for the step size.
    Thanks!

    ReplyDelete
  13. The only other suggestion I have is try the opposite method of capture. The wrapper has two methods: captureFramesWithInterval and captureFramesWithTimeSeek, so try the one you haven't used yet.

    ReplyDelete
  14. Can you give the time seek one multiple times, so give it a comma separated list of all frames every 1' or something? I tried that method before but a problem I had was that it would seem to lock up and get super slow if I tried to call mplayer for each frame separately.
    Thanks!

    ReplyDelete
  15. Contact me off here at boyank at google or the chat, maybe I can help you if you haven't figured it out.

    ReplyDelete
  16. Hi all
    I just found this wrapper and am wondering can I use this to get all the frames from a video file.

    Thanks for any information.

    ReplyDelete
  17. You can but with some modifications to the command line parameters for mplayer.

    ReplyDelete
  18. Thanks for the information
    I am wondering if this command would work
    mplayer input.avi -vo jpeg:outdir=dirname

    or would this be better
    mplayer -ss 00:00:00:01 -vo jpeg input.avi

    I will be working with an avi file

    I found the first command on the mplayer wiki page

    ReplyDelete
  19. Yup, the first one works. However for a 37 seconds movie, I ended up with 949 images, a bit crazy.

    ReplyDelete
  20. Boyan - very helpful code. Thank you! It there a way mplayer.exe can take direct output path as a parameter for jpeg:outdir?

    ReplyDelete
  21. Vladimir, thanks! The mplayer.exe only takes a relative directory for the jpeg output. That's why (if you look at the source for the wrapper) if you want a different output directory, the images are first generated in a relative temp directory and then moved to the final destination.

    ReplyDelete
  22. I am having trouble finding the exact executables and codacs from the mPlayer website, it's pretty confusing. Can you zip everything needed together into a .zip file and post a download link, that way I will be sure to have the right files you created this tutorial using :)

    Thanks a million, happy coding!

    ReplyDelete
  23. I just add dll to my project and copy tour code to the project.......but it produce nothing.........what should i do now please help me

    ReplyDelete
  24. This comment has been removed by a blog administrator.

    ReplyDelete

// //]]>