Audio/Video Sync Problems With Auto-Transcoding

[anna]

Some file formats (particularly wmv and avi) are not being handled well by Indytube/mencoder. The audio is out of sync and the video often appears in fast motion resulting in a 5 minute video lasting only 10 seconds. The audio will continue playing after this. The duration is often reported incorrectly.

Other videos will play more or less normally but the audio will be significantly out of sync.

Here is a version on plumi demo - http://demo.plumi.org/Members/qwerty/videos/cliplarage-medium.wmv/view

And on EM stable http://www.engagemedia.org/Members/AnarchiaFilms/videos/shv-train-track-court-1.mp4/view http://www.engagemedia.org/Members/jonair/videos/nonregularworkersstruggleinnewcoreeland.wmv/view

It seems mencoder has difficulty transcoding files with a variable frame rate. WMV files frequently have a variable frame rate. This may not be the only explanation, but it appears to be the major one.

[anna]

PREVIOUS COMMENTS AS TICKET HAS BEEN MOVED OVER FROM ENGAGEMEDIA.ORG TRACKER:

10/09/07 03:19:01 changed by anna

APEC: 330 Million Dollar Wall of Shame didn't work: Chasers Comedians

http://www.engagemedia.org/Members/thesaint/videos/apec-snipers.wmv/view

http://www.engagemedia.org/author/thesaint

Errors

sound is terrible - very compressed. picture is not synced - rushes past while audio is at normal speed. black frames remainder of video duration is displayed as 00.02, the video plays out to 01.30, but is listed as duration

Half-widows of Kashmir

http://www.engagemedia.org/Members/Halfwidows/videos/parveena-subtitiling-and-operation-snatch.mov/reTranscode

http://www.engagemedia.org/author/Halfwidows

Errors

video ends abruptly after 58seconds

I have changed indytube.conf back to vbitrate=150 instead of 200

I have clicked restart on these videos: http://www.engagemedia.org/Members/thesaint/videos/apec-snipers.wmv/reTranscode http://www.engagemedia.org/Members/Halfwidows/videos/parveena-subtitiling-and-operation-snatch.mov/reTranscode

Will check these tomorrow, document effect of change (if any) and do the same with the others in review queue also.

10/09/07 03:20:19 changed by anna

Note - i killed indytube processes and restarted all instances, otherwise changes to indytube.conf would have had no effect.

10/10/07 02:07:26 changed by anna

APEC: 330 Million Dollar Wall of Shame didn't work: Chasers Comedians

http://www.engagemedia.org/Members/thesaint/videos/apec-snipers.wmv/view

http://www.engagemedia.org/author/thesaint

Errors

sound is terrible - very compressed. picture is not synced - rushes past while audio is at normal speed. black frames remainder of video duration is displayed as 00.02, the video plays out to 01.30, but is listed as duration

Change After Reverting to Old Bitrate

No change. Same error.

10/13/07 16:19:16 changed by andycat

I used ffmpeg on the cmd line , to make the flv here http://testing.engagemedia.org/Members/jonair/videos/nonregularworkersstruggleinnewcoreeland.wmv/view

I think that works ?

I got some feedback from a guy in FOE Amsterdam, who is using indytube, that they have had the same problems as us. He claimed it was a debian issue, with some code missing from the original mplayer sources. I have got a mplater svn checkout built on the server, but so far its not working.

He also said that they use ffmpeg instead of mencoder, as it is more robust. Hence i then explored that, and here is a flv produced using:

ffmpeg -i /opt/instances/zope-testing/var/files/Members/jonair/videos/nonregularworkersstruggleinnewcoreeland.wmv -ab 56 -ar 22050 -b 500 -s 320x240 nonregularworkersstruggleinnewcoreeland.flv

10/14/07 03:32:38 changed by and

That's interesting. Is it possible to transcode the videos in stable by hand that are currently in the review list? A few have been there for more than a month and given how little content we are getting atm it'd be great to add these 5 videos.

Cool that foe are using it. Are they using plumi also? Be great to encourage him to jump on the indytube list so we could stay in the loop. http://lists.codecoop.org/mailman/listinfo/indytube-coders

10/14/07 03:57:13 changed by and

Also this video, which we _assumed_ :) was cut short in transcoding, is actually only 58 seconds long. http://www.engagemedia.org/Members/Halfwidows/videos/parveena-subtitiling-and-operation-snatch.mov/view I will write to them and tell them to upload the full version.

10/21/07 18:40:24 changed by andycat

I have confirmed on my computer that using the latest SVN version of mencoder, and adding -ofps 20 produces a FLV that has the correct video playback speed - although I think there are video/audio sync issues.

related: http://lists.mplayerhq.hu/pipermail/mencoder-users/2007-October/007316.html

10/21/07 19:45:50 changed by andycat

And , in fact , as that post in the url above mentions, the WMV of jonair's video reports FPS of 1000

but i also dont know the implications of hard copying the "-ofps 20" into the mencoder options in indytube - I dont know if the framerate is passed thru , so to speak, from the input video into the FLV.. ?

I believe it is - I ran

root@redfern:/var/www/stable-flv-files# find . -name "*.flv" -exec flvtool2 -P {} \; -print > res

which is the output of the fvltool2 s info about a FLV eg

/var/www/stable-flv-files/Brendan/videos/g20_summit.flv:

audiodatarate: 56.0918489065606 cuePoints: hasKeyframes: true hasVideo: true framerate: 25 canSeekToEnd: false stereo: true lasttimestamp: 100.6 datasize: 3063568 videocodecid: 2 audiosamplerate: 22000 audiosize: 747573 audiosamplesize: 16 videosize: 2314135 audiodelay: 0 hasAudio: true filesize: 3089001 height: 240 lastkeyframetimestamp: 100.2 metadatacreator: inlet media FLVTool2 v1.0.6 - http://www.inlet-media.de/flvtool2 metadatadate: Mon May 14 08:17:47 GMT+0000 2007 duration: 100.64 videodatarate: 181.826640159046 audiocodecid: 2 keyframes:

times:

So doing a search of that file..

root@redfern:/var/www/stable-flv-files# grep framerate res

framerate: 25 framerate: 30 framerate: 25 framerate: 25 framerate: 25 framerate: 30 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 30 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 12 framerate: 25 framerate: 25 framerate: 12 framerate: 25 framerate: 25 framerate: 23 framerate: 23 framerate: 12 framerate: 25 framerate: 10 framerate: 25 framerate: 25 framerate: 2 framerate: 25 framerate: 25 framerate: 25 framerate: 2 framerate: 4 framerate: 26 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 8 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 6 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 12 framerate: 30 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 30 framerate: 25 framerate: 25 framerate: 25 framerate: 14 framerate: 14 framerate: 14 framerate: 14 framerate: 14 framerate: 14 framerate: 14 framerate: 14 framerate: 14 framerate: 14 framerate: 30 framerate: 3 framerate: 25 framerate: 12 framerate: 25 framerate: 25 framerate: 14 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 23 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 30 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 28 framerate: 30 framerate: 25 framerate: 14 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 14 framerate: 25 framerate: 14 framerate: 12 framerate: 25 framerate: 12 framerate: 25 framerate: 25 framerate: 25 framerate: 14 framerate: 14 framerate: 14 framerate: 25 framerate: 12 framerate: 25 framerate: 12 framerate: 25 framerate: 30 framerate: 12 framerate: 10 framerate: 25 framerate: 25 framerate: 12 framerate: 12 framerate: 12 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 2 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 10 framerate: 23 framerate: 25 framerate: 0 framerate: 14 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 14 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 14 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 30 framerate: 25 framerate: 30 framerate: 25 framerate: 25 framerate: 12 framerate: 25 framerate: 25 framerate: 23 framerate: 25 framerate: 25 framerate: 25 framerate: 2 framerate: 4 framerate: 2 framerate: 2 framerate: 14 framerate: 30 framerate: 30 framerate: 25 framerate: 5 framerate: 14 framerate: 30 framerate: 23 framerate: 14 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 30 framerate: 30 framerate: 23 framerate: 30 framerate: 25 framerate: 14 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 30 framerate: 30 framerate: 23 framerate: 23 framerate: 14 framerate: 25 framerate: 25 framerate: 14 framerate: 31 framerate: 12 framerate: 25 framerate: 30 framerate: 14 framerate: 14 framerate: 31 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 1000 framerate: 14 framerate: 25 framerate: 25 framerate: 1000 framerate: 1000 framerate: 23 framerate: 14 framerate: 25 framerate: 25 framerate: 25 framerate: 25 framerate: 10 framerate: 23 framerate: 25 framerate: 1000 framerate: 25 framerate: 25 framerate: 25 framerate: 12 framerate: 12 framerate: 25 framerate: 30 framerate: 25 framerate: 25 framerate: 12

To solve this, we probably need to be smarter about encoding the framerate from the input file.

10/21/07 19:47:33 changed by andycat

http://lists.mplayerhq.hu/pipermail/mencoder-users/2007-October/007348.html

is a thread to watch

10/21/07 20:14:43 changed by andycat

Aah .. http://lists.mplayerhq.hu/pipermail/mencoder-users/2007-March/005534.html 10/21/07 20:18:24 changed by andycat ¶

This is the question we also need answering: http://lists.mplayerhq.hu/pipermail/mencoder-users/2006-October/004364.html 10/22/07 09:13:41 changed by andycat ¶

Links for future reference: converting videos for iPod video using ffmpeg - http://www.julien-oster.de/crash/122.html

http://www.avidemux.org/admForum/viewtopic.php?pid=21862#p21862

http://handbrake.m0k.org/trac/wiki/x264Options

14.7. Using MEncoder to create QuickTime??-compatible files - http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-quicktime-7.html

10/24/07 10:39:54 changed by andycat

Ive added -ofps 30 to indytube.conf on testing

Please test it out on testing.engagemedia.org by retranscoding files, and comparsing them to ones on EM live, esp. the WMVs

10/24/07 11:14:34 changed by andycat

http://testing.engagemedia.org/Members/jonair/videos/nonregularworkersstruggleinnewcoreeland.wmv/view

seems to work on re-transcoding..

To compare between a file that was already working:: http://testing.engagemedia.org/Members/undergrowth/videos/the_black_gst.mov/view http://www.engagemedia.org/Members/undergrowth/videos/the_black_gst.mov/view

Which is think indicates it doesnt work properly. The time on the original track is 12:40 ( user reported - Duration: 12 minutes 41 seconds ) The time on the testing version is 10:33 ( flashplayer)

The metadata is :

/var/www/testing-flv-files/undergrowth/videos/the_black_gst.flv:

audiodatarate: 67.7437495365498 cuePoints: hasKeyframes: true hasVideo: true framerate: 30 canSeekToEnd: false stereo: true lasttimestamp: 633.833 datasize: 18182445 videocodecid: 2 audiosamplerate: 22000 audiosize: 5687576 audiosamplesize: 16 videosize: 12486601 audiodelay: 0 hasAudio: true filesize: 18375002 height: 240 lastkeyframetimestamp: 632.2 metadatacreator: inlet media FLVTool2 v1.0.6 - http://www.inlet-media.de/flvtool2 metadatadate: Wed Oct 24 16:06:04 GMT+0000 2007 duration: 633.866 videodatarate: 154.961007079152 audiocodecid: 2

From the stable version :

/var/www/stable-flv-files/undergrowth/videos/the_black_gst.flv:

audiodatarate: 56.4590332930101 cuePoints: hasKeyframes: true hasVideo: true framerate: 25 canSeekToEnd: true stereo: true lasttimestamp: 760.52 datasize: 25282807 videocodecid: 2 audiosamplerate: 22000 audiosize: 5687576 audiosamplesize: 16 videosize: 19586891 audiodelay: 0 hasAudio: true filesize: 25475356 height: 240 lastkeyframetimestamp: 760.52 metadatacreator: inlet media FLVTool2 v1.0.6 - http://www.inlet-media.de/flvtool2 metadatadate: Thu Sep 20 04:41:32 GMT+0000 2007 duration: 760.56 videodatarate: 203.836711723558 audiocodecid: 2

i just dont know exactly what happens when it transcodes from , say, 24 fps, mpeg or whatever to 30fps FLV

Clearly i think we need to match the FPS to the input. unless its whack like WMV.

10/24/07 11:32:06 changed by andycat

So i remembered about 'harddup' which seems to not drop duplicate frames, and keeps audio and video in sync - from my limited understanding of the situation - and added it into testing's indytube.conf

from "man mencoder"

harddup

Only useful with MEncoder. If harddup is used when encoding, it will force duplicate frames to be encoded in the output. This uses slightly more space, but is necessary for output to MPEG files or if you plan to demux and remux the video stream after encoding. Should be placed at or near the end of the filter chain unless you have a good reason to do otherwise.

This has keep the correct video length of the transcoded file http://testing.engagemedia.org/Members/undergrowth/videos/the_black_gst.mov/view

The new metadata is :

/var/www/testing-flv-files/undergrowth/videos/the_black_gst.flv:

audiodatarate: 56.4605180802104 cuePoints: hasKeyframes: true hasVideo: true framerate: 30 canSeekToEnd: false stereo: true lasttimestamp: 760.5 datasize: 20672825 videocodecid: 2 audiosamplerate: 22000 audiosize: 5687576 audiosamplesize: 16 videosize: 14975559 audiodelay: 0 hasAudio: true filesize: 20880582 height: 240 lastkeyframetimestamp: 758.867 metadatacreator: inlet media FLVTool2 v1.0.6 - http://www.inlet-media.de/flvtool2 metadatadate: Wed Oct 24 16:25:00 GMT+0000 2007 duration: 760.533 videodatarate: 154.893706771861 audiocodecid: 2

So now the video and audo play in sync up to the end! The filesize is SMALLER though - which i find weird, since the video is meant to have more frames in the same length of time.. ?

10/24/07 21:51:35 changed by anna

perhaps when it says "duplicating frames" it merely means in the sense that most lossy compressors do i.e. it has one keyframe, then says repeat that frame for each "duplicate" to maintain sync.

could this explain the smaller file size?

also did you consider the question i raised earlier of whether or not we could preserve native framerates unless they fell below 15 or above 30? is this possible?

10/24/07 21:56:06 changed by anna

also if we are going to force a framerate i would suggest 25 for the engagemedia.org site rather than 30 as 25 is the native PAL framerate which people should be uploading more often.

10/26/07 09:04:23 changed by andycat

From http://gentoo-wiki.com/TIP_MEncoder_Tips_and_Tricks

Encode to NTSC or PAL Framerate

To encode the movie with the NTSC framerate of 29.97 frames per second:

NOTE: 29.97 is an approximation, more accurately, it's 30/1.001 (same applies to 23.976, it's more accurately 24/1.001). New versions of mencoder support fractional values, and thus can be used here

mencoder <filename.avi> -ovc lavc -oac mp3lame -o <output.avi> -ofps 30000/1001

To encode the movie with the PAL framerate of 25 frames per second:

mencoder <filename.avi> -ovc lavc -oac mp3lame -o <output.avi> -ofps 25

Note that you will have A-V sync problems if the framerate of the source does not match the output. See the docs for information about using telecine in these cases.

Telecine is http://en.wikipedia.org/wiki/Telecine

and then somebody needs to digest this i reckon http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-telecine.html :)

11/28/07 00:17:04 changed by anna moving to Plumi tracker.

[anna]

So what to do here? Seems to me there are at least two possible solutions:

1. employ ffmpeg instead of mencoder as from what i have read it is better able to handle A/V sync and variable frame rate input. from andycat's test above this seems to be a good route to take.

2. use "harddup" duplicate frames in mencoder, and also look at the potential for determining the average frame rate (which seems to involve asking mplayer to play the file through at a faster speed and report duration and number of frames, getting a script to determine the average) and passing this back to mencoder so it can create the correct number of frames and maintain sync.

we could ask the script to only go through the process of determining an average frame rate IF the frame rate is >15 or >30 - otherwise pass through the frame rate from the metadata. (apparently mencoder reports the frame rate as 1000 if it is variable)

my research notes are here: http://www.plumi.org/wiki/TranscodingErrors

[and]

Anna and I chatted about this yesterday and though we should bump it up to blocker as EM has been getting a few vids in lately that have had this problem. I realise people know this already but just keeping it at the front of our collective minds.

[anna]

nearly ready to tick this off.. just maybe should re-transcode the EM catalog in order to fully test a variety of formats.

[suparroopxxxh]

Welcome. I'm search in yahoo super ass site.

[url=http://www.reality-kings-sex.com]Reality Kings reviews/url

and

[url=http://www.reality-kings-porn.com]RealityKings[/url]

and

[url=http://www.reality-kings-porno.com]Reality Kings/url

if

[url=http://www.reality-kings-xxx.com]Reality Kings/url

else

[url=http://www.reality-kings-reviews.com]Reality Kings reviews/url

Very thx.If, Goodbye.

[anna]

This has been running for some time now with no problems AFAIK, so ticking this off.