Complete a tutorial on video processing tasks with the Python Django framework _python

Stickyworld Web apps have been supporting video allocations for some time, but they are all implemented via YouTube's embedded model. We are starting to offer new versions to support video operations, allowing our users to not be subject to YouTube services.

I have been involved in a project in the past, the customer needs video transcoding function, this is not an easy to achieve demand. Need to read a lot of each video, audio and video container format and then output in line with the use of Web pages and preferences of the video format.

With this in mind, we decided to hand over the work of transcoding to encoding.com. This site allows you to encode 1GB of video at no charge, and files over 1GB will take a graded pricing charge.

The code developed below, I uploaded a 178KB capacity two seconds video to test whether the code is working successfully. I continue to test other larger external files after the test process has not had any exception errors.

Stage One: Users upload video files

Now this new code snippet provides an upload mechanism that is based on HTML5 and can be used quickly. Code written with Coffeescript can upload files from the client to the server side.

 $scope. Upload_slide = (upload_slide_form)-> file = document.getElementById ("Slide_fi Le "). files[0] reader = new FileReader () reader.readasdataurl File Reader.onload = (event)-> result = Event.ta Rget.result fileName = document.getElementById ("Slide_file"). Files[0].name $.post "/world/upload_slide", data:r Esult name:filename room_id: $scope. Room.id (response_data)-> if response_data.success? Is isn't Yes Console.error "There was a error uploading the file" Response_data else Console.log "Upload
   Successful ", Response_data Reader.onloadstart =-> console.log" Onloadstart "reader.onprogress = (event)-> Console.log "OnProgress", Event.total, event.loaded, (event.loaded/event.total) * Reader.onabort =-> cons Ole.log "onabort" reader.onerror =-> console.log "onerror" Reader.onloadend = (event)-> Console.log "onl" Oadend ", Event 

It is best to pass ("Slide_file"). Files and upload each file via a separate post instead of uploading all files by a post requirement. We will explain this later.

Phase two: Verify and upload to Amazon S3

Back end we ran Django with RABBITMQ. The main modules are as follows:

$ pip Install ' django>=1.5.2 ' django-celery>=3.0.21 ' \ django-storages>=1.1.8 ' lxml>=3.2.3 ' Python-mag
 
ic>=0.4.3 ' I've built two modules: Slideuploadqueue is used to store every uploaded data, and Slidevideomedia is used to store each piece of data to upload the movie. Class Slideuploadqueue (models. Model): Created_by = models. ForeignKey (User) Created_time = models. Datetimefield (db_index=true) Original_file = models. Filefield (Upload_to=filename_sanitiser, Blank=true, default= ') Media_type = models. ForeignKey (mediatype) Encoding_com_tracking_code = models. Charfield (default= ', max_length=24, blank=true) status_awaiting_data = 0 status_awaiting_processing = 1 STAT 
    us_processing = 2 Status_awaiting_3rd_party_processing = 5 Status_finished = 3 Status_failed = 4 STATUS_LIST = ( (Status_awaiting_data, ' awaiting DATA '), (status_awaiting_processing, ' awaiting processing '), (status_process ING, ' processing '), (status_awaiting_3rd_party_processing, ' awaiting 3rd-party processing '), (Status_finished, ' finished '), (status_failed, ' FAILED '), STATUS = models. Positivesmallintegerfield (Default=status_awaiting_data, choices=status_list) class Meta:verbose_name = ' Slid
      E ' verbose_name_plural = ' Slide upload queue ' def save (self, *args, **kwargs): If not self.created_time: Self.created_time = \ Datetime.utcnow (). Replace (TZINFO=PYTZ.UTC) return super (Slideuploadqueue, self). Save (  *args, **kwargs) def __unicode__ (self): If Self.id is None:return ' new <SlideUploadQueue> ' return ' <SlideUploadQueue>%d '% self.id class Slidevideomedia (models. Model): converted_file = models. 
  Filefield (Upload_to=filename_sanitiser, Blank=true, default= ') format_mp4 = 0 FORMAT_WEBM = 1 FORMAT_OGG = 2
    FORMAT_FL9 = 3 Format_thumb = 4 Supported_formats = (Format_mp4, ' MPEG 4 '), (FORMAT_WEBM, ' WEBM '), (Format_ogg, ' OGG '), (FORMAT_FL9, ' Flash 9 video '), (Format_thumb, ' Thumbnail ')), Mime_types = ((Format_mp4, ' Video/mp4 '), (FORMAT_WEBM, ' VIDEO/WEBM '), (Format_ogg, ' Video/ogg '), (FORMAT_FL9, ' Video/mp4 '), (Format_thumb, ' image/jpeg '),) FORMAT = models. Positivesmallintegerfield (Default=format_mp4, choices=supported_formats) class meta:verbose_name = ' Slide VI Deo ' verbose_name_plural = ' Slide videos ' def __unicode__ (self): If Self.id is None:return ' new <sli

 Devideomedia> ' return ' <SlideVideoMedia>%d '% self.id

All of our modules use Filename_sanitiser. Filefield automatically adjusts the file name to <model>/<uuid4>.<extention> format. Organize each filename and ensure its uniqueness. We have adopted a time-sensitive list of URLs that allow us to control which users are using our services for how long.

def filename_sanitiser (instance, filename):
  folder = Instance.__class__.__name__.lower ()
  ext = ' jpg '
 
  if ' . ' In filename:
    t_ext = Filename.split ('. ') [ -1].strip (). Lower ()
 
    if T_ext!= ':
      ext = t_ext return
 
  '%s/%s.%s '% (folder, str (UUID.UUID4 ()), ext)

The file Testing.mov to be tested will be converted to the following URL: https://our-bucket.s3.amazonaws.com/slideuploadqueue/ 3fe27193-e87f-4244-9aa2-66409f70ebd3.mov and uploaded via the Django storages module.

We verify the files uploaded from the user-side browser through Magic. Magic can detect the type of file from the contents of the file.

@verify_auth_token
@return_json
def upload_slide (request):
  File_data = Request. Post.get (' data ', ')
  file_data = Base64.b64decode (File_data.split ('; base64, ') [1])
  description = Magic.from_ Buffer (File_data)

If the file type conforms to the MPEG V4 system or the Apple QuickTime movie, we know that the file transcoding is not too much of a problem. If the format is not mentioned above, we will mark it to the user.
Next, we will store the video to the queue through the Slideuploadqueue module and send a demand to RABBITMQ. Because we use the Django storages module, the file will automatically be uploaded to the Amazon S3.

Slide_upload = Slideuploadqueue () ...
Slide_upload.status = slideuploadqueue.status_awaiting_processing
slide_upload.save ()
Slide_ upload.original_file.\
  Save (' anything.%s '% File_ext, ContentFile (file_data))
Slide_upload.save ()
 
task = Convertrawslidetoslide ()
Task.delay (slide_upload)

Phase 3: Send video to third party.

The

RABBITMQ the call to control Task.delay (slide_upload).
We now only need to send the video file URL with the output format to encoding.com. The site will reply to a work code for us to check the progress of the video transcoding.

Class Convertrawslidetoslide (Task): queue = ' backend_convert_raw_slides ' ... def _handle_video (self, slide_upload):
      mp4 = {' Output ': ' mp4 ', ' size ': ' 320x240 ', ' bitrate ': ' 256k ', ' audio_bitrate ': ' 64k ',
      ' Audio_channels_number ': ' 2 ', ' keep_aspect_ratio ': ' Yes ', ' video_codec ': ' MPEG4 ', ' profile ': ' Main ', ' Vcodecparameters ': ' No ', ' audio_codec ': ' Libfaac ', ' two_pass ': ' No ', ' cbr ': ' No ', ' Deinterlaci
 
    Ng ': ' No ', ' keyframe ': ', ' audio_volume ': ' The ', ' file_extension ': ' mp4 ', ' hint ': ' No ',}
      WEBM = {' Output ': ' webm ', ' size ': ' 320x240 ', ' bitrate ': ' 256k ', ' audio_bitrate ': ' 64k ', ' Audio_sample_rate ': ' 44100 ', ' audio_channels_number ': ' 2 ', ' keep_aspect_ratio ': ' Yes ', ' Video_codec ' : ' libvpx ', ' profile ': ' Baseline ', ' vcodecparameters ': ' No ', ' audio_codec ': ' Libvorbis ', ' Two_pass ': ' No ', ' cbr ': ' NO ', ' deinterlacing ': ' No ', ' keyframe ': ' 6 ', ' audio_volume ': ' m ', ' preset ': ', ' file_ex  Tension ': ' webm ', ' acbr ': ' No ',} ogg = {' Output ': ' Ogg ', ' size ': ' 320x240 ', ' bitrate ': ' 256k ', ' audio_bitrate ': ' 64k ', ' audio_sample_rate ': ' 44100 ', ' audio_channels_number ': ' 2 ', ' Keep
      _aspect_ratio ': ' Yes ', ' video_codec ': ' Libtheora ', ' profile ': ' Baseline ', ' vcodecparameters ': ' No ', ' Audio_codec ': ' Libvorbis ', ' two_pass ': ' No ', ' cbr ': ' No ', ' deinterlacing ': ' No ', ' keyframe ': ' 3 ', ' audio_volume ': ' m ', ' file_extension ': ' Ogg ', ' acbr ': ' No ', ' flv = {' Output ': '
      Fl9 ', ' size ': ' 320x240 ', ' bitrate ': ' 256k ', ' audio_bitrate ': ' 64k ', ' audio_channels_number ': ' 2 ', 
      ' Keep_aspect_ratio ': ' Yes ', ' video_codec ': ' libx264 ', ' profile ': ' High ', ' vcodecparameters ': ' No ', ' Audio_codec ': ' LIbfaac ', ' two_pass ': ' No ', ' cbr ': ' No ', ' deinterlacing ': ' No ', ' keyframe ': ', ' Audio_vol
      Ume ': ', ' file_extension ': ' mp4 ',} thumbnail = {' output ': ' Thumbnail ', ' time ': ' 5 ', ' Video_codec ': ' mjpeg ', ' keep_aspect_ratio ': ' Yes ', ' file_extension ': ' jpg ',} encoder = Encoding ( Settings. encoding_api_user_id, settings. Encoding_api_user_key) resp = Encoder.add_media (Source=[slide_upload.original_file.url), Formats=[mp4, WEBM, OGG , flv, thumbnail]) media_id = None If Resp is not None and resp.get (' response ') are not none:media_id =
      Resp.get (' response '). Get (' MediaID ') if media_id is None:slide_upload.status = slideuploadqueue.status_failed Slide_upload.save () Log.error (' Unable to communicate with encoding.com ') return False slide_upload . Encoding_com_tracking_code = media_id Slide_upload.status = \ Slideuploadqueue.status_awaIting_3rd_party_processing Slide_upload.save () return True

 

Encoding.com recommends some useful Python programs that you can use to communicate with their services. I have modified some parts of the module, but I still need to modify some functions to achieve my satisfactory status. The following is the program code that is currently in use after the modification:

Import httplib from lxml import etree import urllib from Xml.parsers.expat import expaterror import xmltodict Encoding_ Api_url = ' manage.encoding.com:80 ' class Encoding (object): Def __init__ (self, userid, UserKey, Url=encoding_api_url) : Self.url = URL Self.userid = UserID Self.userkey = UserKey def get_media_info (self, action= ' Getmediainfo ', ids=[], headers={' content-type ': ' application/x-www-form-urlencoded '}: query = etree.
      Element (' query ') nodes = {' UserID ': Self.userid, ' UserKey ': self.userkey, ' action ': action, ' MediaID ': ', '. Join (IDS),} query = Self._build_tree (etree.
 
  Element (' query '), nodes) results = self._execute_request (query, headers) return Self._parse_results (results) def get_status (self, action= ' GetStatus ', ids=[], extended= ' no ', headers={' content-type ': ' Application/x-www-form-url Encoded '}): query = etree. Element (' query ') nodes = {' UserID ': Self.userid, ' useRkey ': Self.userkey, ' action ': Action, ' extended ': Extended, ' mediaid ': ', '. Join (IDS),} query = Self._build_tree (etree.
 
  Element (' query '), nodes) results = self._execute_request (query, headers) return Self._parse_results (results) def add_media (self, action= ' Addmedia ', source=[], notify= ', formats=[], instant= ' no ', headers={' content-type ': ' Application/x-www-form-urlencoded '}): query = etree.
      Element (' query ') nodes = {' UserID ': Self.userid, ' UserKey ': self.userkey, ' action ': action, ' Source ': Source, ' Notify ': Notify, ' instant ': instant,} query = Self._build_tree (etree. Element (' query '), nodes) for the format in Formats:format_node = Self._build_tree (etree. Element (' format '), format) query.append (format_node) results = self._execute_request (query, headers) retur n Self._parse_results (results) def _build_tree (Self, node, data): For K, V in Data.items ():
      If Isinstance (V, list): for item in v:element = Etree. Element (k) Element.text = Item Node.append (element) else:element = etree. Element (k) Element.text = v node.append (Element) return node Def _execute_request (self, XML, hea DERs, path= ', method= ' POST '): params = Urllib.urlencode ({' xml ': etree.tostring (XML)}) conn = Httplib. Httpconnection (Self.url) conn.request (method, path, params, headers) response = Conn.getresponse () data = Resp Onse.read () Conn.close () return Data def _parse_results (self, results): Try:return Xmltodict.parse (

 Results) except Expaterror, E:print ' Error parsing encoding.com ' Print e return None

Other pending issues include the use of encoding.com rigorous SSL authentication via Https-only (encrypted online), and some unit tests.
Phase 4: Download all new video file formats

We have a regularly executed program that checks the progress of the video transcoding every 15 seconds by RABBITMQ:

Class Checkuponthirdparties (Periodictask):
  run_every = Timedelta (seconds=settings. Third_party_check_up_interval) ...
  def _handle_encoding_com (self, slides):
    format_lookup = {
      ' mp4 ': slidevideomedia.format_mp4,
      ' WEBM ': SLIDEVIDEOMEDIA.FORMAT_WEBM,
      ' ogg ': Slidevideomedia.format_ogg,
      ' fl9 ': SLIDEVIDEOMEDIA.FORMAT_FL9,
      ' Thumbnail ': Slidevideomedia.format_thumb,
    }
 
    encoder = Encoding (settings. encoding_api_user_id,
      settings. Encoding_api_user_key)
 
    job_ids = [Item.encoding_com_tracking_code for item in slides]
    resp = encoder.get_ Status (Ids=job_ids)
 
    If RESP is None:
      log.error (' Unable to check up in encoding.com ') return
      False

Check the encoding.com response to verify that each part is correct to facilitate our continued.

If Resp.get (' response ') is None:log.error (' Unable to get response node from encoding.com ') return False resp_id = Resp.get (' response '). Get (' id ') if resp_id be None:log.error (' Unable to get media IDs from Encoding.com ') return Fals E slide = SlideUploadQueue.objects.filter (status=slideuploadqueue.status_awaiting_3rd_party_processing, Encoding_c OM_TRACKING_CODE=RESP_ID) If Len (slide)!= 1:log.error (' Unable to find a single record for%s '% resp_id) return Fa LSE Resp_status = Resp.get (' response '). Get (' status ') if Resp_status be None:log.error (' Unable to get status from en Coding.com ') return False if Resp_status!= u ' finished ': Log.debug ("%s isn ' t finished, would check back later"% resp _ID) return True formats = resp.get (' response '). Get (' format ') if formats is None:log.error ("No Output formats Wer E found.
  Something ' s wrong. ") Return False for the format in Formats:try:assert format.get (' status ') = = U ' finished ', \ '%s is not FinisHed. Something ' s wrong.% format.get (' id ') output = format.get (' output ') Assert output in (' mp4 ', ' WEBM ', ' Ogg ', ' FL 9 ', ' thumbnail '), ' Unknown output format%s '% output s3_dest = format.get (' s3_destination ') assert ' http: encoding.com.result.s3.amazonaws.com/' \ In S3_dest, ' suspicious S3 url:%s '% s3_dest = \ ' https://s3.amazonaws.com/encoding.com.result/%s '%\ s3_dest.split ('/') [-1] file_ext = Https_link.split ('. ') [ -1].strip () Assert Len (file_ext) > 0,\ ' Unable to get file extension from%s '% https_link count = s LideVideoMedia.objects.filter (Slide_upload=slide, Format=format_lookup[output]). COUNT () if Count!= 0:p 
 
    Rint ' There is already a%s file for this slide '% output continue content = Self.download_content (https_link)
    Assert content isn't none,\ ' There is no content for%s '% format.get (' id ') except Assertionerror, E: Log.error (' A format didNot pass all assertions:%s '% e) continue

 

Here we have confirmed that everything is OK, so we can store all the video files:

Media = Slidevideomedia ()
Media.format = Format_lookup[output]
media.converted_file.save (' blah.%s '% file_ Ext, ContentFile (content))
Media.save ()

Stage 5: Play the video file via HTML5

On our front page, we've added a page with a HTML5 image unit. The video is displayed using video.js that have the best support for each browser.

? Bower Install Video.js
Bower caching Git://github.com/videojs/video.js-component.git bower cloning git://
Github.com/videojs/video.js-component.git
Bower fetching video.js
Bower checking out video.js#v4.0.3
Bower copying/home/mark/.bower/cache/video.js/5ab058cd60c5615aa38e8e706cd0f307
Bower Installing video.js# 4.0.3

On our homepage There are other dependent files:

!!! 5
html (lang= "en", class= "No-js")
 head
  meta (http-equiv= ' Content-type ', content= ' text/html; charset= UTF-8 ') ...
  Link (rel= ' stylesheet ', type= ' text/css ', href= '/components/video-js-4.1.0/video-js.css ')
  script (type= ' text/ JavaScript ', src= '/components/video-js-4.1.0/video.js ')

In the angular.js/jade-based framework of the module, we introduce <video> volume label and its <source> sub volume label. Each video file will have a thumbnail through the <video> poster component, the thumbnail image is captured by us from the first few seconds of the video.

#main. Span12
  Video#example_video_1.video-js.vjs-default-skin (controls, preload= "Auto", width= "640", height= " 264 ', poster= ' {{video_thumbnail}} ', data-setup= ' {' example_option ': true} ', ng-show= ' videos ')
    source (ng-repeat = "Video in Videos", src= "{{video.src}}", Type= "{{video.type}}")

It will also show the format of each video file we convert and use in <source> tags. Video.js determines which format video is played based on the browser used by the user.

We still have a lot of work to do, set up a program for unit testing to communicate with the Encoding.com service. Please contact me if you are interested in the work.