GStreamer Basic Tutorial 13--playback speed

Goal

Fast forward, rewind and slow down are common tricks of the trick pattern, and they have one thing in common: they all change the speed of playback. This tutorial shows you how to get these effects and how to jump frames by frame. The main contents are:

How to change the speed of play, become faster or slower, forward or backward

How to play a frame-by-frame video

Introduced

Fast Forward is a technology that plays media faster than normal speed, whereas slow play is a technique that plays at a lower speed than normal. The rewind and playback are the same, but they are played from the back towards the front.

All of these techniques are to modify the playback speed of the matter, if the normal playback speed is 1.0, then more than 1.0 this number is fast forward, less than 1.0 this number is slow to put, positive is from the back of the former, negative numbers are from the back ahead of the release.

GStreamer provides two ways to change the speed of playback: The step event and the Seek event. The step event can skip a specified interval (which can only be played forward) with the subsequent playback speed changed. Seek event, you can jump to any place and you can set the playback speed (positive reverse).

The Seek event has been demonstrated in the GStreamer Basic tutorial 04--time management, using a helper function to hide complexity. This tutorial will explain in more detail.

The step event is more convenient to change the playback speed because it requires fewer parameters. However, they need to do a little more work in the implementation of GStreamer, so the Seek event is used here.

In order to use these events, they need to be established and then passed on to pipeline, which will propagate upstream to the element that can handle these events. If an event is passed to a bin element (such as playbin2), it simply gives the event to all of its sink, which can cause the operation to execute many times. It is common practice to find a playbin2 sink through the Video-sink or Audio-sink attribute and pass the event directly to the sink.

Step by frame is a frame-by-frame playback of the video, which is to let pipeline pause, then send the step event to it, and let it jump one frame at a time.

A magic mode player

#include <string.h>
#include <gst/gst.h>
  
typedef struct _CustomData {
  GstElement *pipeline;
  GstElement *video_sink;
  GMainLoop *loop;
  
  gboolean playing;  /* Playing or Paused */
  gdouble rate;      /* Current playback rate (can be negative) */
} CustomData;
  
/* Send seek event to change rate */
static void send_seek_event (CustomData *data) {
  gint64 position;
  GstFormat format = GST_FORMAT_TIME;
  GstEvent *seek_event;
  
  /* Obtain the current position, needed for the seek event */
  if (!gst_element_query_position (data->pipeline, &format, &position)) {
    g_printerr ("Unable to retrieve current position.\n");
    return;
  }
  
  /* Create the seek event */
  if (data->rate > 0) {
    seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
        GST_SEEK_TYPE_SET, position, GST_SEEK_TYPE_SET, -1);
  } else {
    seek_event = gst_event_new_seek (data->rate, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE,
        GST_SEEK_TYPE_SET, 0, GST_SEEK_TYPE_SET, position);
  }
  
  if (data->video_sink == NULL) {
    /* If we have not done so, obtain the sink through which we will send the seek events */
    g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
  }
  
  /* Send the event */
  gst_element_send_event (data->video_sink, seek_event);
  
  g_print ("Current rate: %g\n", data->rate);
}
  
/* Process keyboard input */
static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomData *data) {
  gchar *str = NULL;
  
  if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) != G_IO_STATUS_NORMAL) {
    return TRUE;
  }
  
  switch (g_ascii_tolower (str[0])) {
  case 'p':
    data->playing = !data->playing;
    gst_element_set_state (data->pipeline, data->playing ? GST_STATE_PLAYING : GST_STATE_PAUSED);
    g_print ("Setting state to %s\n", data->playing ? "PLAYING" : "PAUSE");
    break;
  case 's':
    if (g_ascii_isupper (str[0])) {
      data->rate *= 2.0;
    } else {
      data->rate /= 2.0;
    }
    send_seek_event (data);
    break;
  case 'd':
    data->rate *= -1.0;
    send_seek_event (data);
    break;
  case 'n':
    if (data->video_sink == NULL) {
      /* If we have not done so, obtain the sink through which we will send the step events */
      g_object_get (data->pipeline, "video-sink", &data->video_sink, NULL);
    }
    
    gst_element_send_event (data->video_sink,
        gst_event_new_step (GST_FORMAT_BUFFERS, 1, data->rate, TRUE, FALSE));
    g_print ("Stepping one frame\n");
    break;
  case 'q':
    g_main_loop_quit (data->loop);
    break;
  default:
    break;
  }
  
  g_free (str);
  
  return TRUE;
}
  
int main(int argc, char *argv[]) {
  CustomData data;
  GstStateChangeReturn ret;
  GIOChannel *io_stdin;
  
  /* Initialize GStreamer */
  gst_init (&argc, &argv);
  
  /* Initialize our data structure */
  memset (&data, 0, sizeof (data));
  
  /* Print usage map */
  g_print (
    "USAGE: Choose one of the following options, then press enter:\n"
    " 'P' to toggle between PAUSE and PLAY\n"
    " 'S' to increase playback speed, 's' to decrease playback speed\n"
    " 'D' to toggle playback direction\n"
    " 'N' to move to next frame (in the current direction, better in PAUSE)\n"
    " 'Q' to quit\n");
  
  /* Build the pipeline */
  data.pipeline = gst_parse_launch ("playbin2 uri=http://docs.gstreamer.com/media/sintel_trailer-480p.webm", NULL);
  
  /* Add a keyboard watch so we get notified of keystrokes */
#ifdef _WIN32
  io_stdin = g_io_channel_win32_new_fd (fileno (stdin));
#else
  io_stdin = g_io_channel_unix_new (fileno (stdin));
#endif
  g_io_add_watch (io_stdin, G_IO_IN, (GIOFunc)handle_keyboard, &data);
  
  /* Start playing */
  ret = gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
  if (ret == GST_STATE_CHANGE_FAILURE) {
    g_printerr ("Unable to set the pipeline to the playing state.\n");
    gst_object_unref (data.pipeline);
    return -1;
  }
  data.playing = TRUE;
  data.rate = 1.0;
  
  /* Create a GLib Main Loop and set it to run */
  data.loop = g_main_loop_new (NULL, FALSE);
  g_main_loop_run (data.loop);
  
  /* Free resources */
  g_main_loop_unref (data.loop);
  g_io_channel_unref (io_stdin);
  gst_element_set_state (data.pipeline, GST_STATE_NULL);
  if (data.video_sink != NULL)
    gst_object_unref (data.video_sink);
  gst_object_unref (data.pipeline);
  return 0;
}

Work flow

The initialization code inside the main function doesn't have anything new: Initialize a playbin2, track the keys, and run a glib main loop.

Then, in the keyboard handler function:

/* Process Keyboard input *
/static Gboolean Handle_keyboard (Giochannel *source, giocondition cond, CustomData *data) {
  Gchar *str = NULL;
  
  if (G_io_channel_read_line (source, &str, NULL, NULL, NULL)! = G_io_status_normal) {
    return TRUE;
  }
  
  Switch (G_ascii_tolower (str[0])) {case
  ' P ':
    data->playing =!data->playing;
    Gst_element_set_state (Data->pipeline, data->playing?) gst_state_playing:gst_state_paused);
    G_print ("Setting state to%s\n", data->playing?) "PLAYING": "PAUSE");
    Break

Use Gst_element_set_state () to handle pause/play alternation as in the previous lecture.

  Case ' s ':
    if (G_ascii_isupper (Str[0])) {
      data->rate *= 2.0;
    } else {
      data->rate/= 2.0;
    }
    Send_seek_event (data);
    break;
  Case ' d ':
    data->rate *= -1.0;
    Send_seek_event (data);
    Break

Use ' s ' to double the speed of playback, ' s ' to reduce the playback speed by one and use ' d ' to convert the playback direction. In any of these cases, the rate variable needs to be updated and then called the Send_seek_event () method. Let's take a look at the function:

/* Send seek event to change rate *
/static void Send_seek_event (CustomData *data) {
  gint64 position;
  Gstformat format = gst_format_time;
  Gstevent *seek_event;
  
  /* Obtain the current position, needed for the Seek event *
  /if (!gst_element_query_position (Data->pipeline, & (format, &position)) {
    G_printerr ("Unable to retrieve current position.\n");
    return;
  }

This function creates a new seek time and sends it to pipeline for a higher playback speed. First, use the Gst_element_query_position () method to record the current position. This is done because the seek event jumps elsewhere, and if we don't want to move later, we need to return to the original position. The step event will be a little simpler, but the step event is not yet fully completed, as described earlier.

  /* Create The Seek event *
  /if (Data->rate > 0) {
    seek_event = Gst_event_new_seek (Data->rate, Gst_format _time, Gst_seek_flag_flush | Gst_seek_flag_accurate,
        gst_seek_type_set, Position, Gst_seek_type_set,-1);
  } else {
    seek_event = Gst_event_new_seek (Data->rate, gst_format_time, Gst_seek_flag_flush | Gst_seek_flag_accurate,
        gst_seek_type_set, 0, gst_seek_type_set, position);
  }

We use Gst_event_new_seek () to establish the Seek event. The parameters are basically the new playback speed, the new start position and the new end position. Regardless of the direction of playback, the starting position is before the end position. So the two playback directions are divided into two segments.

  if (Data->video_sink = = NULL) {/
    * If we have no do so, obtain the sink through which we'll send the seek even TS *
    /G_object_get (Data->pipeline, "Video-sink", &data->video_sink, NULL);
  }

As explained earlier, in order to avoid this seek, the event is only passed to a sink, here, is the video sink. Obtained by Playbin2 's Video-sink property. This action executes here instead of at initialization because the sink is different with the playback media, so video-sink is not deterministic until pipeline to playing state and has read some media data.

  /* Send the event *
  /Gst_element_send_event (Data->video_sink, seek_event);

Finally, we use Gst_element_send_event () to pass the newly created event to the selected sink.

Back to the keyboard handler function, we haven't mentioned the frame stepping code that was implemented here:

  Case ' n ':
    if (data->video_sink = = NULL) {/
      * If we have no do so, obtain the sink through which we'll send The Step events *
      /G_object_get (Data->pipeline, "Video-sink", &data->video_sink, NULL);
    }
    
    Gst_element_send_event (Data->video_sink,
        gst_event_new_step (gst_format_buffers, 1, data->rate, TRUE, FALSE));
    G_print ("Stepping One frame\n");
    Break

Using Gst_event_new_step () to create a new step event, the parameters are mainly the specified step size (here is 1 frames) and the speed (we did not change here).

Take a look at Playbin2 's video sink, as mentioned above.

Done. However, when testing this tutorial, keep in mind that playback is not supported by many element.

(You can also modify the playback speed for local files, if you want to experiment with this, simply change the URL passed to playbin2 to a local URL.) Please note that the file:///is used as the beginning)