Desktop Java

Xuggler Tutorial: Frames Capture and Video Creation

Note: This is part of our “Xuggler Development Tutorials” series.

So far in our Xuggler tutorials series we have performed an Introduction to Xuggler for Video Manipulation and we have discussed Transcoding and Media Modification. In this tutorial, we shall see how to decode video and capture frames, as well as how to create video from scratch.

Let’s begin with decoding a video stream and capturing some frames at predefined time intervals. This can be used, for example, in order to make thumbnails of a media file. For this purpose, we will use again the MediaTool API, a high-level API for decoding, encoding and modifying video.

The concept is to open the media file, loop through a specific video stream and at specific intervals capture the corresponding frame, convert it to an image and dump the binary contents into a file. Here is what the code for all these looks like:

package com.javacodegeeks.xuggler;

import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;

import javax.imageio.ImageIO;

import com.xuggle.mediatool.IMediaReader;
import com.xuggle.mediatool.MediaListenerAdapter;
import com.xuggle.mediatool.ToolFactory;
import com.xuggle.mediatool.event.IVideoPictureEvent;
import com.xuggle.xuggler.Global;

public class VideoThumbnailsExample {
    
    public static final double SECONDS_BETWEEN_FRAMES = 10;

    private static final String inputFilename = "c:/Java_is_Everywhere.mp4";
    private static final String outputFilePrefix = "c:/snapshots/mysnapshot";
    
    // The video stream index, used to ensure we display frames from one and
    // only one video stream from the media container.
    private static int mVideoStreamIndex = -1;
    
    // Time of last frame write
    private static long mLastPtsWrite = Global.NO_PTS;
    
    public static final long MICRO_SECONDS_BETWEEN_FRAMES = 
        (long)(Global.DEFAULT_PTS_PER_SECOND * SECONDS_BETWEEN_FRAMES);

    public static void main(String[] args) {

        IMediaReader mediaReader = ToolFactory.makeReader(inputFilename);

        // stipulate that we want BufferedImages created in BGR 24bit color space
        mediaReader.setBufferedImageTypeToGenerate(BufferedImage.TYPE_3BYTE_BGR);
        
        mediaReader.addListener(new ImageSnapListener());

        // read out the contents of the media file and
        // dispatch events to the attached listener
        while (mediaReader.readPacket() == null) ;

    }

    private static class ImageSnapListener extends MediaListenerAdapter {

        public void onVideoPicture(IVideoPictureEvent event) {

            if (event.getStreamIndex() != mVideoStreamIndex) {
                // if the selected video stream id is not yet set, go ahead an
                // select this lucky video stream
                if (mVideoStreamIndex == -1)
                    mVideoStreamIndex = event.getStreamIndex();
                // no need to show frames from this video stream
                else
                    return;
            }

            // if uninitialized, back date mLastPtsWrite to get the very first frame
            if (mLastPtsWrite == Global.NO_PTS)
                mLastPtsWrite = event.getTimeStamp() - MICRO_SECONDS_BETWEEN_FRAMES;

            // if it's time to write the next frame
            if (event.getTimeStamp() - mLastPtsWrite >= 
                    MICRO_SECONDS_BETWEEN_FRAMES) {
                                
                String outputFilename = dumpImageToFile(event.getImage());

                // indicate file written
                double seconds = ((double) event.getTimeStamp()) / 
                    Global.DEFAULT_PTS_PER_SECOND;
                System.out.printf(
                        "at elapsed time of %6.3f seconds wrote: %s\n",
                        seconds, outputFilename);

                // update last write time
                mLastPtsWrite += MICRO_SECONDS_BETWEEN_FRAMES;
            }

        }
        
        private String dumpImageToFile(BufferedImage image) {
            try {
                String outputFilename = outputFilePrefix + 
                     System.currentTimeMillis() + ".png";
                ImageIO.write(image, "png", new File(outputFilename));
                return outputFilename;
            } 
            catch (IOException e) {
                e.printStackTrace();
                return null;
            }
        }

    }

}

This might seem a bit overwhelming, but it is really quite straightforward. Let me provide some details for you. We start by creating an IMediaReader from an input file. The media reader is used to read and decode media. Since we wish to manipulate the captures video frames as images, we use the setBufferedImageTypeToGenerate method to denote this. The reader opens up a media container, reads packets from it, decodes the data, and then dispatches information about the data to any registered IMediaListener objects. Here is where our custom class, ImageSnapListener, comes into play.

Our listener extends the MediaListenerAdapter, which is an adapter (provides empty methods) implementing the IMediaListener interface. Objects that implement this interface are notified about events generated during the video processing. We only care about handling video events, thus we only implement the IMediaListener.onVideoPicture method. Inside that we use the provided IVideoPictureEvent object to find what stream (video only) we are dealing with.

Since we wish to capture frames in specific times, we have to mess a little with timestamps. First, we make sure we handle the very first frame by checking against the value of the Global.NO_PTS constant, which is a value that means no time stamp is set for a given object. Then, if the minimum elapsed time has passes, we capture the frame by invoking the IVideoPictureEvent.getImage method, which returns the underlying BufferedImage. Note that we are talking about elapsed video time and not “real time”. We then dump the image data to a file in PNG format using the ImageIO.write utility method. Finally, we update the last write time.

Let’s run this application in order to see the results. As input file, I am using an old Sun commercial proclaiming that “Java is Everywhere”. I have downloaded locally the MP4 version provided. Here is what the output console will look like:

at elapsed time of 0.000 seconds wrote: c:/snapshots/mysnapshot1298228503292.png
at elapsed time of 10.010 seconds wrote: c:/snapshots/mysnapshot1298228504014.png
at elapsed time of 20.020 seconds wrote: c:/snapshots/mysnapshot1298228504463.png

at elapsed time of 130.063 seconds wrote: c:/snapshots/mysnapshot1298228509454.png
at elapsed time of 140.007 seconds wrote: c:/snapshots/mysnapshot1298228509933.png
at elapsed time of 150.017 seconds wrote: c:/snapshots/mysnapshot1298228510379.png

The total video time is about 151 seconds so we capture 16 frames. Here is what the captured images look like in my folder:

OK, that’s about it for making video thumbnails. Let’s now see how to create a video from scratch. As input, we will use sequential snapshots from our desktop. This can be used for a rudimentary screen recording application.

In order to create video, we will have to take a bit more low level approach in comparison to the MediaTool API that we have seen so far. Don’t worry though, it is not going to be complicated. The main idea is that we create a media writer, add some stream information to it, encode our media (the screenshot images), and close the writer. Let’s see the code used to achieve this:

package com.javacodegeeks.xuggler;

import java.awt.AWTException;
import java.awt.Dimension;
import java.awt.Rectangle;
import java.awt.Robot;
import java.awt.Toolkit;
import java.awt.image.BufferedImage;
import java.util.concurrent.TimeUnit;

import com.xuggle.mediatool.IMediaWriter;
import com.xuggle.mediatool.ToolFactory;
import com.xuggle.xuggler.ICodec;

public class ScreenRecordingExample {
    
    private static final double FRAME_RATE = 50;
    
    private static final int SECONDS_TO_RUN_FOR = 20;
    
    private static final String outputFilename = "c:/mydesktop.mp4";
    
    private static Dimension screenBounds;

    public static void main(String[] args) {

        // let's make a IMediaWriter to write the file.
        final IMediaWriter writer = ToolFactory.makeWriter(outputFilename);
        
        screenBounds = Toolkit.getDefaultToolkit().getScreenSize();

        // We tell it we're going to add one video stream, with id 0,
        // at position 0, and that it will have a fixed frame rate of FRAME_RATE.
        writer.addVideoStream(0, 0, ICodec.ID.CODEC_ID_MPEG4, 
                   screenBounds.width/2, screenBounds.height/2);

        long startTime = System.nanoTime();
        
        for (int index = 0; index < SECONDS_TO_RUN_FOR * FRAME_RATE; index++) {
            
            // take the screen shot
            BufferedImage screen = getDesktopScreenshot();

            // convert to the right image type
            BufferedImage bgrScreen = convertToType(screen, 
                   BufferedImage.TYPE_3BYTE_BGR);

            // encode the image to stream #0
            writer.encodeVideo(0, bgrScreen, System.nanoTime() - startTime, 
                   TimeUnit.NANOSECONDS);

            // sleep for frame rate milliseconds
            try {
                Thread.sleep((long) (1000 / FRAME_RATE));
            } 
            catch (InterruptedException e) {
                // ignore
            }
            
        }
        
        // tell the writer to close and write the trailer if  needed
        writer.close();

    }
    
    public static BufferedImage convertToType(BufferedImage sourceImage, int targetType) {
        
        BufferedImage image;

        // if the source image is already the target type, return the source image
        if (sourceImage.getType() == targetType) {
            image = sourceImage;
        }
        // otherwise create a new image of the target type and draw the new image
        else {
            image = new BufferedImage(sourceImage.getWidth(), 
                 sourceImage.getHeight(), targetType);
            image.getGraphics().drawImage(sourceImage, 0, 0, null);
        }

        return image;
        
    }
    
    private static BufferedImage getDesktopScreenshot() {
        try {
            Robot robot = new Robot();
            Rectangle captureSize = new Rectangle(screenBounds);
            return robot.createScreenCapture(captureSize);
        } 
        catch (AWTException e) {
            e.printStackTrace();
            return null;
        }
        
    }

}

We start by creating an IMediaWriter from a given output file. This class encodes and decodes media, handling both audio and video streams. Xuggler guesses the output format from the file name extension (in our case MP4) and sets some default values appropriately. We then use the addVideoStream method to add a new video stream, providing its index, the codec type used (MPEG-4 here) and the video dimensions. The dimensions are set equal to half of the screen’s dimensions in this example.

Then we execute a loop that runs for a number of times equal to the desired frame rate multiplied with the desired running time. Inside the loop, we generate a screen snapshot as described in the Java2D: Screenshots with Java article. We retrieve the screenshot as a BufferedImage and convert it to the appropriate type (TYPE_3BYTE_BGR) if it is not already there.

Next, we encode the image to the video stream using the IMediaWriter.encodeVideo method. We provide the stream index, the image, the elapsed video time and the time unit. Then, we sleep for the appropriate number of time, depending on the desired frame rate. When the loop is over, we close the writer and write the trailer if necessary, depending on the video format (this is done automatically by Xuggler).

If we execute the application, a video will be created which has recorded your desktop actions. Here is a still image of mine while browsing the JavaCodeGeeks site:

That’s it guys, yet another Xuggler tutorial describing how to capture video frames from an input file and how to generate video using desktop snapshots. As always, you can download the Eclipse project created for this tutorial. Stay tuned for more Xuggler tutorials here at JavaCodeGeeks! And don’t forget to share!

Related Articles:

Ilias Tsagklis

Ilias is a software developer turned online entrepreneur. He is co-founder and Executive Editor at Java Code Geeks.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

46 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Raja Sekaran
12 years ago

Error: Could not find or load main class VideoThumbnailsExample

Gianfranco Garipoli
Gianfranco Garipoli
12 years ago
Reply to  Raja Sekaran

did you add the jars libraries to the build path?

Punit Saini
11 years ago

can u please tell me which jar i need to add?
please share download links

Mahesh Saini
11 years ago

yes also get same error can u give me download link to these jars

nirmalnk
nirmalnk
9 years ago
Reply to  Raja Sekaran

DO maven clean, maven install
then update project

Gianfranco Garipoli
Gianfranco Garipoli
12 years ago

Hi
thanks! everything works!
i have one question
What about the audio stream?? i would like to store also the audio of this video. Because after making some changing to all the pictures i would like to re-make the video again with the new pictures.

thanks

akilan thamarai
akilan thamarai
12 years ago

can u give ur  mail id for contact and i have some questions pz.
it will very helpful for my project
my id is akilan.cricket@gmail.com

Raji ac
Raji ac
11 years ago

this program is not giving any result..i have used  xuggle-xuggler-3.0.660.jar API

Mahesh Saini
11 years ago

I got the flowing errors
java.lang.NoClassDefFoundError: VideoThumbnailsExample (wrong name: com/javacodegeeks/xuggler/VideoThumbnailsExample)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(Unknown Source)
at java.lang.ClassLoader.defineClass(Unknown Source)
at java.security.SecureClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.access$000(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
Could not find the main class: VideoThumbnailsExample. Program will exit.
Exception in thread “main”

can any body help me

sagar
sagar
11 years ago
Reply to  Mahesh Saini

heyy same problem with me also. please help me to solve this one.

sallu
sallu
11 years ago

WOW … great tutorial … .its save my life …. i am searching for this program from long time … THANK’s

oussama
oussama
10 years ago
Reply to  sallu

hello,

How can I save/rename the frames this way:

frame_1.png
frame_2.png
frame_3.png

(And not like that:

mysnapshot1298228503292.png
mysnapshot1298228504014.png
)

Thanks.

Fajar Suryawan
11 years ago

I’d like to have this: a desktop app that has two panels. One is a live view from a camera (desirably with a level of control over the zoom and autofocus). The other one is a sill jpeg image that is generated from the first panel when an operator click the “Take a Snapshot” button. How do I acheive this? Thanks!

Ted
Ted
10 years ago

I was wondering if you have a tutorial that can make a video from a sequences of images.

sylvio
sylvio
10 years ago
Reply to  Ted

instead of using sanpshots use an array of images

Mayank
Mayank
10 years ago

I am getting following error:

[main] ERROR com.xuggle.ferry.JNILibraryLoader – Could not load library: xuggle-xuggler; version: 3; Visit http://www.xuggle.com/xuggler/faq/ to find common solutions to this problem
java.lang.UnsatisfiedLinkError: no xuggle-xuggler in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)
at java.lang.Runtime.loadLibrary0(Runtime.java:845)
at java.lang.System.loadLibrary(System.java:1084)
at com.xuggle.ferry.JNILibraryLoader.loadLibrary0(JNILibraryLoader.java:265)
at com.xuggle.ferry.JNILibraryLoader.loadLibrary(JNILibraryLoader.java:168)
at com.xuggle.xuggler.XugglerJNI.(XugglerJNI.java:19)
at com.xuggle.xuggler.Global.(Global.java:238)
at xug.VideoThumbnailsExample.(VideoThumbnailsExample.java:28)
Exception in thread “main” Java Result: 1

oussama
oussama
10 years ago
Reply to  Mayank

Hello Mayank,

you must have the latest version of xuggler. You can download all necessary jars from this link:
http://www.benfarahmand.com/2012/11/tutorial-using-xuggler-in-processing.html#!/2012/11/tutorial-using-xuggler-in-processing.html look at “step2”).
Good luck.

Hareesh
Hareesh
9 years ago
Reply to  oussama

There is no downloads oussama, so please provide another download links for new jar files

oussama
oussama
10 years ago

I am getting following error:

SLF4J built for org.slf4j.impl.JDK14LoggerFA
java.lang.NoSuchMethodError: org.slf4j.Logger.trace(Ljava/lang/String;)V
at com.xuggle.ferry.JNILibraryLoader.(JNILibraryLoader.java:235)
at com.xuggle.ferry.JNILibraryLoader.(JNILibraryLoader.java:189)
at com.xuggle.xuggler.XugglerJNI.(XugglerJNI.java:19)
at com.xuggle.xuggler.Global.(Global.java:238)
at com.javacodegeeks.xuggler.VideoThumbnailsExample.(VideoThumbnailsExample.java:27)
Exception in thread “main” Java Result: 1

oussama
oussama
10 years ago

Hello Everybody,

How can I save/rename the frames this way:

frame_1.png
frame_2.png
frame_3.png

(And not like that: mysnapshot1298228503292.png
mysnapshot1298228504014.png
mysnapshot1298228504463.png
…..
)

I’m waiting for your urgent response.
Thanks.

Lobotomia
Lobotomia
9 years ago
Reply to  oussama

– insert at line 16
private static int counter = 1;

– change line 20 (private static final String outputFilePrefix = “c:/snapshots/mysnapshot”;)
private static final String outputFilePrefix = “c:/snapshots/frame_”;

– change line 86 (String outputFilename = outputFilePrefix + System.currentTimeMillis() + “.png”;)
String outputFilename = outputFilePrefix + counter + “.png”;

– insert at line 87
counter++;

mks
mks
10 years ago

how can we do this same thing by upload video in website and grab and frame and save this image to database

puneet
puneet
10 years ago

is it possible to record video/audio without xuggler/ffmpeg/server side in java applet. I cannot use xuggler as it has 40 mb and takes tool long time for loading

Nitin
Nitin
9 years ago

2014-04-08 20:49:21,614 [main] ERROR com.xuggle.ferry.JNILibraryLoader – Could not load library: xuggle-xuggler; version: 3; Visit http://www.xuggle.com/xuggler/faq/ to find common solutions to this problem
Exception in thread “main” java.lang.UnsatisfiedLinkError: no xuggle-xuggler in java.library.path
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
at java.lang.System.loadLibrary(Unknown Source)
at com.xuggle.ferry.JNILibraryLoader.loadLibrary0(JNILibraryLoader.java:265)
at com.xuggle.ferry.JNILibraryLoader.loadLibrary(JNILibraryLoader.java:168)
at com.xuggle.xuggler.XugglerJNI.(XugglerJNI.java:19)
at com.xuggle.xuggler.Global.(Global.java:238)
at com.javacodegeeks.xuggler.VideoThumbnailsExample.(VideoThumbnailsExample.java:27)

Im getting this exception, and i have included jar etc. .. when i run this at eclipse i get this, please any suggestions?

Hareesh
Hareesh
9 years ago

[main] ERROR com.xuggle.ferry.JNILibraryLoader – Could not load library: xuggle-xuggler; version: 3; Visit http://www.xuggle.com/xuggler/faq/ to find common solutions to this problem

it shows the error so please help me to slove this.

Hareesh
Hareesh
9 years ago

please give me reply as soon as possible..

Raj Makwana
Raj Makwana
9 years ago

java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at com.xuggle.ferry.JNILibraryLoader.(JNILibraryLoader.java:48)
at com.xuggle.xuggler.XugglerJNI.(XugglerJNI.java:19)
at com.xuggle.xuggler.Global.(Global.java:238)
at projecttrain.VideoThumbnailsExample.(VideoThumbnailsExample.java:45)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)

How can I solve this error ? Please Help

Balaji
9 years ago

following errors i am facing while running the above program…please help me ASAP

Exception in thread “main” java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at com.xuggle.ferry.JNILibraryLoader.(JNILibraryLoader.java:48)
at com.xuggle.xuggler.XugglerJNI.(XugglerJNI.java:19)
at com.xuggle.xuggler.Global.(Global.java:238)
at com.balaji.videotoimageconvert.VideoThumbnailsExample.(VideoThumbnailsExample.java:25)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
… 4 more

Raj
Raj
9 years ago

How can i solve this error…. :

java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at com.xuggle.ferry.JNILibraryLoader.(JNILibraryLoader.java:48)
at com.xuggle.xuggler.XugglerJNI.(XugglerJNI.java:19)
at com.xuggle.xuggler.Global.(Global.java:238)
at projecttrain.VideoThumbnailsExample.(VideoThumbnailsExample.java:45)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)

Carlos
Carlos
9 years ago

For anyone having “java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory” problem, this is the solution (worked for me):

http://stackoverflow.com/questions/13434862/xuggle-error-exception-in-thread-main-java-lang-noclassdeffounderror-org-slf

ayesha
ayesha
9 years ago

[main] ERROR com.xuggle.ferry.JNILibraryLoader – Could not load library: xuggle-xuggler; version: 3; Visit http://www.xuggle.com/xuggler/faq/ to find common solutions to this problem
how to solve this problem

Back to top button