GStreamer iOS 教程2 —— 運行pipeline

1. 目標

在Basic和Playback的教程中（註：另外兩個教程），GStreamer可以和GLib的主迴圈完美的整合起來，這樣能用一種簡單的方法同時監控pipeline的操作和UI的處理。而在不支援GLib的iOS和Android平台上就必須小心對於pipeline的操作——不能阻塞住UI。

這份教程講述了：

如何把GStreamer的相關處理代碼放到其他的線程（DispatchQueue），保持UI仍然保留在主線程（MainDispatchQueue）在ObjC的UI代碼中如何和GStreamer的C代碼通訊2. 介紹 當由UI介面的時候，如果應用等待GStreamer的回傳訊息然後進行UI的處理是非常悲催的。通常的做法是（用GTK＋toolkit做例子哈）讓GMainLoop（GLib）來處理收到的event，無論是UI的還是GStreamer發出的。 悲催的是這個方法不適合其他非基於GLib的圖形系統（比如iOS的GocoaTouch架構），我們的方法是在另一個線程裡面簡單的調用GMainLoop，確保不會阻塞UI主線程。 這個教程還會指出幾個ObjC和C互相調用時需要注意的幾個地方。 下面的代碼使用的pipeline僅僅使用了audiotestsrc和autoaudiosink兩個element。UI上包含兩個按鈕，用來設定pipeline的狀態PLAYING/ PAUSED。還有一個Label用來顯示一些資訊（錯誤資訊或狀態改變）。
3. UI 這個介面底下包含了一個工具條，工具條上放了Play和Pause兩個按鈕。工具條上方就是一個Label，用來顯示GStreamer的資訊。本次教程就包含這麼多內容，後面的教程會在這個基礎上逐漸增加內容。ViewController.m

#import "ViewController.h"#import "GStreamerBackend.h"#import @interface ViewController () {    GStreamerBackend *gst_backend;}@end@implementation ViewController/* * Methods from UIViewController */- (void)viewDidLoad{    [super viewDidLoad];        play_button.enabled = FALSE;    pause_button.enabled = FALSE;    gst_backend = [[GStreamerBackend alloc] init:self];}- (void)didReceiveMemoryWarning{    [super didReceiveMemoryWarning];    // Dispose of any resources that can be recreated.}/* Called when the Play button is pressed */-(IBAction) play:(id)sender{    [gst_backend play];}/* Called when the Pause button is pressed */-(IBAction) pause:(id)sender{    [gst_backend pause];}/* * Methods from GstreamerBackendDelegate */-(void) gstreamerInitialized{    dispatch_async(dispatch_get_main_queue(), ^{        play_button.enabled = TRUE;        pause_button.enabled = TRUE;        message_label.text = @"Ready";    });}-(void) gstreamerSetUIMessage:(NSString *)message{    dispatch_async(dispatch_get_main_queue(), ^{        message_label.text = message;    });}@end

在這個類裡麵包含了一個GStreamerBackend的執行個體，

@interface ViewController () {    GStreamerBackend *gst_backend;}

在viewDidLoad方法裡面建立並調用了自訂的init方法

- (void)viewDidLoad{    [super viewDidLoad];        play_button.enabled = FALSE;    pause_button.enabled = FALSE;    gst_backend = [[GStreamerBackend alloc] init:self];}

這個自訂的init方法必須傳入一個對象作為UI的delegate（本例是使用了self）
在viewDidLoad的時候，Play/Pause兩個按鈕都是不能使用的，直到GStreamerBackend通知初始化結束為止。

/* Called when the Play button is pressed */-(IBAction) play:(id)sender{    [gst_backend play];}/* Called when the Pause button is pressed */-(IBAction) pause:(id)sender{    [gst_backend pause];}
      在使用者按下Play/Pause按鈕時，上面的方法會被調用。我們看到僅僅就是簡單的調用了GStreamerBackend裡面對應的方法。
-(void) gstreamerInitialized{    dispatch_async(dispatch_get_main_queue(), ^{        play_button.enabled = TRUE;        pause_button.enabled = TRUE;        message_label.text = @"Ready";    });}
     gstreamerInitialized方法是定義在GStreamerBackendDelegate協議裡面的，用來標識後台已經準備好，可以接受命令了。在這個例子中我們把Play/Pause按鈕啟用，並顯示“Ready”資訊。這個方法不是在UI線程裡面啟動並執行，所以要用dispatch_async()方法把UI的內容封起來。
-(void) gstreamerSetUIMessage:(NSString *)message{    dispatch_async(dispatch_get_main_queue(), ^{        message_label.text = message;    });}
     gstreamerSetUIMessage:方法同樣是定義在GStreamerBackendDelegate協議裡面的，當後台有訊息通知UI的時候會調用這個方法。在這個例子裡面訊息會拷貝到介面的Label控制項上，當然，也同樣要用dispatch_async()方法來封裝。
4. GStreamer後端      GStreamerBackend類處理了所有和GStreamer相關的內容，並給應用提供了一個簡單的介面。這個介面並不需要實現所有的GStreamer的細節，當需要引起UI的變化的時候，調用GSreamerBackendDelegate協議來解決。GStreamerBackend.m
#import "GStreamerBackend.h"#include GST_DEBUG_CATEGORY_STATIC (debug_category);#define GST_CAT_DEFAULT debug_category@interface GStreamerBackend()-(void)setUIMessage:(gchar*) message;-(void)app_function;-(void)check_initialization_complete;@end@implementation GStreamerBackend {    id ui_delegate;        /* Class that we use to interact with the user interface */    GstElement *pipeline;  /* The running pipeline */    GMainContext *context; /* GLib context used to run the main loop */    GMainLoop *main_loop;  /* GLib main loop */    gboolean initialized;  /* To avoid informing the UI multiple times about the initialization */}/* * Interface methods */-(id) init:(id) uiDelegate{    if (self = [super init])    {        self->ui_delegate = uiDelegate;        GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "iOS tutorial 2");        gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);        /* Start the bus monitoring task */        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{            [self app_function];        });    }    return self;}-(void) dealloc{    if (pipeline) {        GST_DEBUG("Setting the pipeline to NULL");        gst_element_set_state(pipeline, GST_STATE_NULL);        gst_object_unref(pipeline);        pipeline = NULL;    }}-(void) play{    if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {        [self setUIMessage:"Failed to set pipeline to playing"];    }}-(void) pause{    if(gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {        [self setUIMessage:"Failed to set pipeline to paused"];    }}/* * Private methods *//* Change the message on the UI through the UI delegate */-(void)setUIMessage:(gchar*) message{    NSString *string = [NSString stringWithUTF8String:message];    if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])    {        [ui_delegate gstreamerSetUIMessage:string];    }}/* Retrieve errors from the bus and show them on the UI */static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self){    GError *err;    gchar *debug_info;    gchar *message_string;        gst_message_parse_error (msg, &err, &debug_info);    message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);    g_clear_error (&err);    g_free (debug_info);    [self setUIMessage:message_string];    g_free (message_string);    gst_element_set_state (self->pipeline, GST_STATE_NULL);}/* Notify UI about pipeline state changes */static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self){    GstState old_state, new_state, pending_state;    gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);    /* Only pay attention to messages coming from the pipeline, not its children */    if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {        gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));        [self setUIMessage:message];        g_free (message);    }}/* Check if all conditions are met to report GStreamer as initialized. * These conditions will change depending on the application */-(void) check_initialization_complete{    if (!initialized && main_loop) {        GST_DEBUG ("Initialization complete, notifying application.");        if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])        {            [ui_delegate gstreamerInitialized];        }        initialized = TRUE;    }}/* Main method for the bus monitoring code */-(void) app_function{    GstBus *bus;    GSource *bus_source;    GError *error = NULL;    GST_DEBUG ("Creating pipeline");    /* Create our own GLib Main Context and make it the default one */    context = g_main_context_new ();    g_main_context_push_thread_default(context);        /* Build pipeline */    pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);    if (error) {        gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);        g_clear_error (&error);        [self setUIMessage:message];        g_free (message);        return;    }        /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */    bus = gst_element_get_bus (pipeline);    bus_source = gst_bus_create_watch (bus);    g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);    g_source_attach (bus_source, context);    g_source_unref (bus_source);    g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);    g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);    gst_object_unref (bus);        /* Create a GLib Main Loop and set it to run */    GST_DEBUG ("Entering main loop...");    main_loop = g_main_loop_new (context, FALSE);    [self check_initialization_complete];    g_main_loop_run (main_loop);    GST_DEBUG ("Exited main loop");    g_main_loop_unref (main_loop);    main_loop = NULL;        /* Free resources */    g_main_context_pop_thread_default(context);    g_main_context_unref (context);    gst_element_set_state (pipeline, GST_STATE_NULL);    gst_object_unref (pipeline);        return;}@end
      其中，介面方法是：
-(id) init:(id) uiDelegate{    if (self = [super init])    {        self->ui_delegate = uiDelegate;        GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "iOS tutorial 2");        gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);        /* Start the bus monitoring task */        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{            [self app_function];        });    }    return self;}
      這個init方法通過調用［super init］來產生執行個體，儲存delegate的對象用來做UI互動，接著調用app_function並運行在一個獨立的並發的線程裡面，app_function會一直監聽GStreamer匯流排，看看有沒有應用需要處理的訊息或者警告發生。      init:方法同樣註冊了一個新的GStreamer調試類別並設定了吐出的資訊的等級，我們就可以在Xcode裡面看到列印資訊了。
-(void) dealloc{    if (pipeline) {        GST_DEBUG("Setting the pipeline to NULL");        gst_element_set_state(pipeline, GST_STATE_NULL);        gst_object_unref(pipeline);        pipeline = NULL;    }}
      dealloc方法把pipeline置成NULL狀態並釋放它。
-(void) play{    if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {        [self setUIMessage:"Failed to set pipeline to playing"];    }}-(void) pause{    if(gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {        [self setUIMessage:"Failed to set pipeline to paused"];    }}
    play/pause方法僅僅簡單的設定pipeline狀態的該變，並在出錯的時候通知UI    下面是檔案中幾個私人方法：
-(void)setUIMessage:(gchar*) message{    NSString *string = [NSString stringWithUTF8String:message];    if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])    {        [ui_delegate gstreamerSetUIMessage:string];    }}
      setUIMessage:方法是把GStreamer使用的C的字串轉變成NSString＊字串，然後調用GStreamerBackendProtocal協議裡面的gstreamerSetUIMessage:方法來在螢幕上顯示出來。      因為這個方法是optional的，所以需要用respondsToSelector來判一下是否存在。
/* Retrieve errors from the bus and show them on the UI */static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self){    GError *err;    gchar *debug_info;    gchar *message_string;        gst_message_parse_error (msg, &err, &debug_info);    message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);    g_clear_error (&err);    g_free (debug_info);    [self setUIMessage:message_string];    g_free (message_string);    gst_element_set_state (self->pipeline, GST_STATE_NULL);}/* Notify UI about pipeline state changes */static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self){    GstState old_state, new_state, pending_state;    gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);    /* Only pay attention to messages coming from the pipeline, not its children */    if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {        gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));        [self setUIMessage:message];        g_free (message);    }}
      error_cb()和state_changed_cb()是註冊的兩個回調，分別在GStreamer出錯和狀態變化的時候被調用。這兩個回調的目的是當事件發生時能通知到使用者。      這兩個回呼函數在Base的教程中出現了多次，實現起來也除了下面2點之外基本一致：一個是訊息使用私人方法setUIMessage:來傳遞到UI；第二個是要調用setUIMessage:就需要一個GStreamerBackend的執行個體，通過callback的userdata來傳遞，這個在下面討論app_function裡回調的註冊時可以看到
-(void) check_initialization_complete{    if (!initialized && main_loop) {        GST_DEBUG ("Initialization complete, notifying application.");        if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])        {            [ui_delegate gstreamerInitialized];        }        initialized = TRUE;    }}
      check_initialization_complete()方法確認滿足所有的條件之後通知UI後台GStreamer準備完成。在這個教程裡面這個條件非常簡單，僅僅是主迴圈存在並且沒有通知過UI。後續的教程這裡會更加複雜。      絕大部分的GStreamer的行為都是在app_function裡面實現的，這些代碼和android的教程幾乎一致。
    /* Create our own GLib Main Context and make it the default one */    context = g_main_context_new ();    g_main_context_push_thread_default(context);
      這裡第一次建立了一個GLib的上下文，使用g_main_context_new()，然後用g_main_context_push_thread_default()來建立了一個線程

    /* Build pipeline */    pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);    if (error) {        gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);        g_clear_error (&error);        [self setUIMessage:message];        g_free (message);        return;    }
      這裡用gst_arse_launch()方法很輕易的建立了一個pipeline。在這個教程裡面僅僅audiotestsrc和autoaudiosink兩個element需要完成適配。
    /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */    bus = gst_element_get_bus (pipeline);    bus_source = gst_bus_create_watch (bus);    g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);    g_source_attach (bus_source, context);    g_source_unref (bus_source);    g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);    g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);    gst_object_unref (bus);
      這幾行建立了一個匯流排訊號的監視器，並和設定了需要監視的訊號，這些和Basic教程裡面的做法也是一致的。這個監視器我們這裡是一步一步建立的，並非調用gst_bus_add_signal_watch()來建立，這樣可以看清如何使用GLib的一些內容。這裡需要指出的是使用了__bridge來把一個ObjC的對象指標轉換成C語言裡面的指標。
      這裡我們不需要太擔心對象的所有權的轉移問題，因為在回調裡面userdata會把這個指標帶回來，重新轉換成GStreamerBackend的對象指標
    /* Create a GLib Main Loop and set it to run */    GST_DEBUG ("Entering main loop...");    main_loop = g_main_loop_new (context, FALSE);    [self check_initialization_complete];    g_main_loop_run (main_loop);    GST_DEBUG ("Exited main loop");    g_main_loop_unref (main_loop);    main_loop = NULL;
      最後，主迴圈建立並開始運行，在進入主迴圈之前，我們調用了check_initialization_complete()方法。主迴圈會一直運行，直到退出為止。
      這篇教程有點長了，主要是需要講清楚一系列的基礎內容。在這個基礎之上，後面的會比較短一些，並且會只關注新的內容。
5. 結論
      這篇教程主要講述了：
使用DispatchQueue如何讓GStreamer的代碼單獨運行在子線程中如何在ObjC的UI代碼和GStreamer的C代碼中傳遞對象
      在這篇教程裡面的方法，象check_initialization_complete()和app_function()，init:，play;，pause:，gstreamerInitialized:和setUIMessage:等介面後續會簡單修改一下繼續使用，所以最好熟悉一下。