Large Video Conversion - Time Out

Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Quote: 01/26/15 07:41:06PM 157.55.39.97 deleted video_conversions queue entry running for 3666 seconds - 3rd time failing

This is a MP4 video file 314 MB in size and about 36 minutes long. The client is in one of my paid quotas, so I want to allow this, he's paying for the server resources.

I have conversion set to the highest level with 1 worker.

1) With 2 workers will it cut the time in half, or just allow 2 files to be processed at once? (Should server upgrade be faster processor or more processors).

2) Where can I increase the 3666 second time cap, so that it doesn't keep giving up. It tried 3 times which means 3 hours of processing 1 hour each. All wasted. If it was set to 3 hours to start with it, it might have finished and not been wasted resources.

3) Did it delete the file for real, or just from the queue? I see it (or at least a record of it) in the datastore. However, I see no way of adding it back to the queue.

4) My host says he has a media conversion package on the server that converts videos really quickly. He's wondering why it's so slow through the script?


--

Ken Rich
indiegospel.net

updated by @ken-rich: 03/14/15 10:33:40PM
brian
@brian
9 years ago
10,148 posts
Ken_Rich:
Quote: 01/26/15 07:41:06PM 157.55.39.97 deleted video_conversions queue entry running for 3666 seconds - 3rd time failing

This is a MP4 video file 314 MB in size and about 36 minutes long. The client is in one of my paid quotas, so I want to allow this, he's paying for the server resources.

I have conversion set to the highest level with 1 worker.

1) With 2 workers will it cut the time in half, or just allow 2 files to be processed at once? (Should server upgrade be faster processor or more processors).

It won't cut the time in half, but will allow 2 conversions going on at once.

Quote:
2) Where can I increase the 3666 second time cap, so that it doesn't keep giving up. It tried 3 times which means 3 hours of processing 1 hour each. All wasted. If it was set to 3 hours to start with it, it might have finished and not been wasted resources.

It looks like the video worker is using the "default" timeout - we'll need to set that higher in the module - probably set it to 4 hours - if it takes longer than that , the server is too slow or the video too big.

Quote:
3) Did it delete the file for real, or just from the queue? I see it (or at least a record of it) in the datastore. However, I see no way of adding it back to the queue.

It gave up on it - the video will need to be deleted and re-uploaded for it to create the queue entry again.

Quote:
4) My host says he has a media conversion package on the server that converts videos really quickly. He's wondering why it's so slow through the script?

Jamroom uses ffmpeg which is pretty much the "standard" for video and audio conversion. If your hosting provider has some super charged version of it, you can modify your data/config/config.php file and add:

$_conf['jrCore_ffmpeg_binary'] = '/usr/local/bin/ffmpeg';

change /usr/local/bin/ffmpeg to the location of your custom ffmpeg. I don't think it will be a lot faster though.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Quote:
It looks like the video worker is using the "default" timeout - we'll need to set that higher in the module - probably set it to 4 hours - if it takes longer than that , the server is too slow or the video too big.

Hi Brian,

Thanks for all that info. Is there a place where I can change that or do I wait for an update?


--

Ken Rich
indiegospel.net
brian
@brian
9 years ago
10,148 posts
Ken_Rich:
Quote:
It looks like the video worker is using the "default" timeout - we'll need to set that higher in the module - probably set it to 4 hours - if it takes longer than that , the server is too slow or the video too big.

Hi Brian,

Thanks for all that info. Is there a place where I can change that or do I wait for an update?

You'll have to wait for an update - it's part of the queue worker definition. I'll get that up here for you as soon as I can.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
OK - thanks


--

Ken Rich
indiegospel.net
brian
@brian
9 years ago
10,148 posts
Ken_Rich:
OK - thanks

I just pushed out version 1.2.3 that ups the timeout to 4 hours - update and you should be set.

Hope this helps!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Strumelia
Strumelia
@strumelia
9 years ago
3,603 posts
This is outrageous! Ken had to wait a whole NINE MINUTES for that module update???

;D


--
...just another satisfied Jamroom customer.
Migrated from Ning to Jamroom June 2015
paul
@paul
9 years ago
4,326 posts
Strumelia:
This is outrageous! Ken had to wait a whole NINE MINUTES for that module update???

;D

Yeah - Sorry about that, we will try harder next time ;-)


--
Paul Asher - JR Developer and System Import Specialist
SteveX
SteveX
@ultrajam
9 years ago
2,584 posts
paul:
Strumelia:
This is outrageous! Ken had to wait a whole NINE MINUTES for that module update???

;D

Yeah - Sorry about that, we will try harder next time ;-)

I need my timeout to be 3.78 hours. Lots of people will love that and then Jamroom will be famous. ;)


--
¯\_(ツ)_/¯ Education, learning resources, TEL, AR/VR/MR, CC licensed content, panoramas, interactive narrative, sectional modules (like jrDocs), lunch at Uni of Bristol. Get in touch if you share my current interests or can suggest better :)
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Wow - yes that was fast - lol. I only asked because I didn't know if I was supposed to be looking for something to tweak.

Should be getting server hardware upgrade this week which will speed things up.

Host says he has this video package installed, but it all seems to be related to streaming not conversion.
Lighttpd Streaming
Lighttpd web server with mod_flv_streaming installed.
Apache mod_flvx Video Streaming
Apache web server with mod_flvx installed.
Apache mod_h264 Video Streaming (HD)
Apache web server with mod_h264 installed.
RTMP Video Streaming
Adobe Flash Media Server installed with the streaming directory pointed to the script video directory.


--

Ken Rich
indiegospel.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
SteveX:
I need my timeout to be 3.78 hours. Lots of people will love that and then Jamroom will be famous. ;)

You have a singular wit - lol


--

Ken Rich
indiegospel.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Wow - guess what. The dang thing timed out again even at that limit.

The server company is asking what kind of settings the script is using for conversion and if we have memory coaching - "memcached" or "memcache".

He's also checking with the data center to see if there is a bottleneck hardware wise. I'm set to priority and overclocked, so it's not possible for him to assign anymore "juice".

I'm going to try a small vid just to see if something hasn't gone wrong with the conversion process altogether. I had tested before and a 3-5 minute music video, perhaps 30-50 MB in size converts in about 10 minutes.


--

Ken Rich
indiegospel.net
brian
@brian
9 years ago
10,148 posts
Ken_Rich:
Wow - guess what. The dang thing timed out again even at that limit.

The server company is asking what kind of settings the script is using for conversion and if we have memory coaching - "memcached" or "memcache".

This has nothing to do with conversions, so ignore them when they ask that.

Quote:
He's also checking with the data center to see if there is a bottleneck hardware wise. I'm set to priority and overclocked, so it's not possible for him to assign anymore "juice".

I'm going to try a small vid just to see if something hasn't gone wrong with the conversion process altogether. I had tested before and a 3-5 minute music video, perhaps 30-50 MB in size converts in about 10 minutes.

Yeah if it is taking LONGER than 4 hours, either the video is extremely corrupted (and ffmpeg is basically "resetting itself" during frame scans many times per second) or your server CPU is just REALLY slow.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
brian:
This has nothing to do with conversions, so ignore them when they ask that.

My mistake in wording it in a way that would make you assume that. He is wondering if the script uses memory coaching - "memcached" or "memcache" but not in relation to conversion.

The CPU is the same one used in your speed test baseline, so should do a reasonable job.

I didn't think of file corruption but perhaps that's it. I'll experiment and try to find out.


--

Ken Rich
indiegospel.net
brian
@brian
9 years ago
10,148 posts
Ken_Rich:
My mistake in wording it in a way that would make you assume that. He is wondering if the script uses memory coaching - "memcached" or "memcache" but not in relation to conversion.

Jamroom only uses memcache as part of the Proxima Cache module - normal Jamroom uses MySQL for caching (so yes - there is definitely caching in JR - A LOT of caching).

Quote:
The CPU is the same one used in your speed test baseline, so should do a reasonable job.

I didn't think of file corruption but perhaps that's it. I'll experiment and try to find out.

Then it should convert it pretty quickly. If you can place the video somewhere where I can download it, I will run it through our dedicated conversion service here and see if I see issues.

Thanks!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Hi Brian,

That would be extremely helpful - thanks. You can download the video here:
https://docs.google.com/file/d/0BzjzgYbFj5_gQlRnTmJwOTVOVUU/edit

I'll try converting some smaller ones tomorrow.


--

Ken Rich
indiegospel.net
brian
@brian
9 years ago
10,148 posts
Thanks for posting the video Ken - I just ran it through our converter here and it worked:

(conversion id 36774) video_file for BrianTest (315MB, m4v) completed in 2463.02 seconds

took 41 minutes and 5 seconds to process, but it had no problems, so I'm suspecting your CPU speed or something else on your server that is keeping it from converting.

Hope this helps!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Thanks Brian, that's really helpful.

The host says he can process folders full of videos with the native conversion package really quickly, so I'm still puzzling on this one.

I'm testing with some small ones downloaded from youtube, I'll let you know how I make out.


--

Ken Rich
indiegospel.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
I downloaded a small video off of youtube and uploaded for conversion. It was a 3 minute MP4 at 15.4MB. It took about 5 minutes to convert.

The host thinks the scripts converter must be misconfigured or something, because according to him, his native converter flies though small files like that in probably under a minute.

Question: Is that a reasonable time for this script, given the size and format of the file (details attached).

Message [Randy Ward]: converted 554/332/video_file from mp4 in 307.63 seconds
Date 01/30/15 07:12:55PM
IP Address xxxxxxxxxx
URL /core/form_validate/__ajax=1
Memory 30MB
Data
yt:quotatoo_many_recent_calls
Capturey.JPG.jpg
Capturey.JPG.jpg  •  22KB




--

Ken Rich
indiegospel.net

updated by @ken-rich: 01/30/15 03:23:41PM
brian
@brian
9 years ago
10,148 posts
That sound about right for a 15.4 mb file, so I don't think that is bad. One thing that is important to know is that your hosting provider is likely just converting the video file ONE time. The JR conversion process:

- converts the video to FLV (H264) for desktop and some android phones
- converts the video to an MP4 for mobile
- grabs a screenshot from the video

If there was an actual issue with JR conversions, we would see it here on our converter and we don't, so my guess is your server just isn't that fast (and based on how fast kenrich.me loaded here for me when I checked out your site I would say your server is on the slow side).

Not sure if that helps though...


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

updated by @brian: 01/30/15 03:44:28PM
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Hi Brian,

That's useful information - thanks.

I don't know why the site loaded slow for you, unless there is a bad node between you and it, the performance test Jamroom provides says the server is fast (see attached).
Capture2.JPG.jpg
Capture2.JPG.jpg  •  68KB




--

Ken Rich
indiegospel.net

updated by @ken-rich: 01/30/15 04:01:57PM
brian
@brian
9 years ago
10,148 posts
Yep that looks good - running traceroute to kenrich.me doesn't work very well for me from here:

 1  192.168.1.1 (192.168.1.1)  2.095 ms  1.053 ms  0.893 ms
 2  static-50-46-224-1.evrt.wa.frontiernet.net (50.46.224.1)  3.016 ms  3.654 ms  2.836 ms
 3  50.35.7.185 (50.35.7.185)  3.719 ms  3.295 ms  7.877 ms
 4  ae3---0.cor02.sttl.wa.frontiernet.net (74.40.1.101)  4.820 ms  4.221 ms  6.614 ms
 5  ae1---0.cbr01.sttl.wa.frontiernet.net (74.40.5.126)  5.220 ms  4.596 ms  5.055 ms
 6  twtelecom.com (206.81.80.64)  5.429 ms  5.966 ms  6.015 ms
 7  orl1-ar3-xe-1-0-0-0.us.twtelecom.net (66.192.243.202)  80.775 ms  86.136 ms  82.935 ms
 8  64.132.248.142 (64.132.248.142)  89.320 ms  143.063 ms  202.052 ms
 9  unassigned-ptr.infinitumtech.net (66.35.95.118)  89.785 ms  80.188 ms *
10  * * *
11  * * *
12  * * *
13  * * *

Looks like a lot of dropped packets, although it could be the routers are not setup to respond to ICMP echo requests. But it is slow to your site for me from here.

Your performance check looks good though, so I'm not sure what to recommend as far as conversions go.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
brian:
That sound about right for a 15.4 mb file, so I don't think that is bad. One thing that is important to know is that your hosting provider is likely just converting the video file ONE time. The JR conversion process:
- converts the video to FLV (H264) for desktop and some android phones
- converts the video to an MP4 for mobile
- grabs a screenshot from the video

The host's converter does the same - FLV (H264) and MP4, plus it grabs 8 thumbnails. It just does it approximately 5 times faster (by current estimates) - on the same hardware.

Whatever configuration settings exist regarding JR's conversion process, are set by the script itself. I have it set to highest, that's all I have control over.

So at this point, I don't know what to think. Something has to account for the huge discrepancy in speed. I'm going to test his converter with a know quantity, for an exact comparison.

Also, I managed to convert an 11 minute long 98.4 MB MP4. However, it didn't leave a log entry and it took over an hour.


--

Ken Rich
indiegospel.net
brian
@brian
9 years ago
10,148 posts
OK let me know what you find out. Thanks!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Hi Brian,

Just a quick note cause I'm late to be somewhere. I noticed today that the lowest and highest setting on the video conversion don't make much of a discernible difference, at least on a small file.

That may be the issue, yesterday when I had it converting a large video on highest setting, the host was watching the sever and it wasn't working. It wasn't "grabbing" a core and working it hard, even though at the server level it's set "full throttle". It wasn't even breaking a sweat.

If the script's settings are choking it, is there a way to "open it up". Maybe after highest, another level. Perhaps called "insane - full throttle - use at own risk."


--

Ken Rich
indiegospel.net

updated by @ken-rich: 01/31/15 01:45:32PM
brian
@brian
9 years ago
10,148 posts
Ken_Rich:
Hi Brian,

Just a quick note cause I'm late to be somewhere. I noticed today that the lowest and highest setting on the video conversion don't make much of a discernible difference, at least on a small file.

That may be the issue, yesterday when I had it converting a large video on highest setting, the host was watching the sever and it wasn't working. It wasn't "grabbing" a core and working it hard, even though at the server level it's set "full throttle". It wasn't even breaking a sweat.

If the script's settings are choking it, is there a way to "open it up". Maybe after highest, another level. Perhaps called "insane - full throttle - use at own risk."

When I get back to my office I'll see there is a command line ffmpeg switch we're not using that could help out. I'll let you know.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
brian
@brian
9 years ago
10,148 posts
One thing that would be helpful is if you could provide me the ffmpeg command line you are using that is 5 times faster than what JR is doing - I'd like to see what switches are being used.

Thanks!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
It's not actually 5 times faster. I did an objective test, by getting him to convert a video that took me 5 minutes 20 seconds (at highest setting). He did it in 1 minute 45 seconds. So using the same file, it's actually about 3 times faster.

I don't know if his converter is FFMPEG, he said he had these packages (below) installed. I'm just wondering if there is a way to let the JR converter run faster. The script controls the settings, and according to his observations, the server (core) is not working hard during conversion.

Lighttpd Streaming
Lighttpd web server with mod_flv_streaming installed.
Apache mod_flvx Video Streaming
Apache web server with mod_flvx installed.
Apache mod_h264 Video Streaming (HD)
Apache web server with mod_h264 installed.
RTMP Video Streaming
Adobe Flash Media Server installed with the streaming directory pointed to the script video directory.


--

Ken Rich
indiegospel.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
derrickhand300:
I hired someone to try and fix this- I don't completely understand what they did but it does seem they convert 3X faster now and I can upload widescreen and have them output as widescreen
Still having a problem with large videos failing though ( 400mb)
The changes he made were in the flv.php file-I will try to post it below ( he did something to do with "ultrafast" and changed (lowered the audio quality) Hope this helps

// make sure we are not being called directly
defined('APP_DIR') or exit();
function jrVideo_flv_decode($input_file,$_options,$error_file)
{
return $input_file;
}
function jrVideo_flv_encode($input_file,$_options,$error_file)
{
$ffmpeg = jrVideo_get_ffmpeg_command();

ob_start();
system("{$ffmpeg} -y -i \"{$input_file}\" -threads ". jrVideo_get_ffmpeg_thread() ." -f flv -acodec libmp3lame -vf scale=640:-2 -ar 22050 -ab 128k -vcodec libx264 -preset ultrafast -qp 0 \"{$input_file}.flv\" >/dev/null 2>{$error_file}",$ret);
ob_end_clean();
return "{$input_file}.flv";
}

Wow Derrick - thanks a lot. First of all, for confirmation that yes there does appear to be a speed issue here. Also, for supplying the fix you paid for - mighty nice of you.

I'm not sure what all that "gobble-de-goop" means, but I'm willing to bet Brian does, and can probably implement it for everyone in an update.


--

Ken Rich
indiegospel.net

updated by @ken-rich: 02/01/15 05:43:57AM
brian
@brian
9 years ago
10,148 posts
Thanks for posting this - I will do some comparisons here and move these changes in as an option if they work well.

Thanks!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Hi Derrick,

I looked at this and compared it to the original in modules/jrVideo-release-1.2.3/plugins/flv.php. This snippet is the important part.
flv -acodec libmp3lame -vf scale=640:-2 -ar 22050 -ab 128k -vcodec libx264 -preset ultrafast -qp 0

I'm certainly no expert, but from what I can gather this is what the changes do.

-vf scale=640:-2 was added - which is probably what solved the widescreen to 4:3 problem.

-preset veryfast to -preset ultrafast - probably gave most of your speed boost. It increases speed by sacrificing compression. So your files take up more space on the server.

-ar 44,100 to -ar 22050 halves the audio sample frequency. Probably most noticeable on the "crispness" of the hi hat and such, but I agree most people wouldn't tell the difference. However, I don't see that as being the big speed boost.

-qp 0 was added. A write up I found says constant quantizer mode. Generally should not be used, since CRF gives better quality at the same bitrate. CRF defaults to 23 when not specified which is a reasonable balance between speed/compression/quality.

===================================

I'm wondering if you are not making a sacrifice in quality to increase speed, due to a separate issue. If I am understanding correctly, video conversion is extremely resource intensive, so a core being used for it should be registering high usage on a performance monitor.

However, my sever guy tells me when I am converting, my CPU usage is minimal, and that is controlled by the script. So perhaps we should be looking for commands that allow the CPU core to work harder to decrease time, rather than sacrificing quality (or compression) to decrease time.

I see there exists a command called -threads and another called -cpu which would have an effect on how the CPU is utilized, but I'm not competent enough to suggest settings.

===================================

Another method to approach this (according to some write ups), might be to simply copy over some files (when they are compatible with the container). Not sure if or how that would work in JR.
Quote:
An MP4 file normally contains H.264 video and AAC audio, both of which are compatible with the FLV container. You could simply copy over the streams:
ffmpeg -i input.mp4 -c copy output.flv
Since you're copying the streams, this will be as close to instantaneous as you're going to get and will not result in any loss of quality.



--

Ken Rich
indiegospel.net

updated by @ken-rich: 02/01/15 09:09:12AM
brian
@brian
9 years ago
10,148 posts
Ken_Rich:
However, my sever guy tells me when I am converting, my CPU usage is minimal, and that is controlled by the script.

When you have the Video Conversion Priority set to the highest, it will use a threads value of "0", which disables confining the encoding to a single CPU - so the load is spread across as many CPUs in your system. Set the value to "high" instead and you should see 1 CPU be pegged.

Quote:
Another method to approach this (according to some write ups), might be to simply copy over some files (when they are compatible with the container). Not sure if or how that would work in JR.
Quote:
An MP4 file normally contains H.264 video and AAC audio, both of which are compatible with the FLV container. You could simply copy over the streams:
ffmpeg -i input.mp4 -c copy output.flv
Since you're copying the streams, this will be as close to instantaneous as you're going to get and will not result in any loss of quality.

Right now JR does a reconversion with -sameq, so the quality is always matched, but ensures everything is in the correct format. This is something we could investigate if it really presented itself as an issue.

I'd rather not go down the "rabbit hole" of changing ffmpeg settings unless they offer a significant speed AND quality advantage (i.e. I'd rather have the encoding be a bit slower and produce a smaller and higher quality file - at the end of the day disk space matters more).

However, there's still an issue here with your server - our conversions appear to run much, much faster (over 4 times faster) than on your server, so something is limiting you on your end.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

updated by @brian: 02/01/15 09:33:24AM
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
brian:
However, there's still an issue here with your server - our conversions appear to run much, much faster (over 4 times faster) than on your server, so something is limiting you on your end.

Hi Brian,

On your other points, I have no problem.

However, as you saw (attached above), according to Jamroom's own performance tests - my speed is good. So in order for what you say to be true, your conversion server would have to be a "monster" built 4 times faster than the baseline for your performance tests (and beyond what most people can afford).

Also, the host checked my settings and I have no performance limits, caps, or bottlenecks of any sort. At least nothing imposed by the server. He says the conversion settings are all controlled by the script, so how can the issue be on "my end".

When converting, my maximum CPU usage is only 15.7 % of what the server is capable of giving me, and sometimes it's "idle", so why isn't the converter utilizing what's available?

To my mind, that is the real issue here. What limits the CPU usage? The host says that's all controlled by the script.


--

Ken Rich
indiegospel.net

updated by @ken-rich: 02/01/15 10:59:36AM
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
derrickhand300:
I didn't really understand what he did but he explained everything comes at a sacrifice somewhere else.

I work from a 32 inch monitor here and after the changes I can tell no difference in video quality.
The biggest thing I noticed is that a (Im guessing now) 15-20 mb file uploaded- when it hit the conversion phases it only lasted "seconds" and not minutes"- where it was taking several minutes before it only seems to take 15-20 seconds ( it really popped)
After looking at the videos on my monitor and thinking 70% of my site visitors will be using much smaller mobile devices I decided it was a sacrifice worth making, at least until I get the site up and get some feedback.

Wow - that's an impressive performance increase. Also, if you can't see or hear the difference - why not?


--

Ken Rich
indiegospel.net
brian
@brian
9 years ago
10,148 posts
Ken_Rich:
brian:
However, there's still an issue here with your server - our conversions appear to run much, much faster (over 4 times faster) than on your server, so something is limiting you on your end.

However, as you saw (attached above), according to Jamroom's own performance tests - my speed is good. So in order for what you say to be true, your conversion server would have to be a "monster" built 4 times faster than the baseline for your performance tests (and beyond what most people can afford).

There's a lot more involved in audio and video conversion than you'll get out of the performance test. I converted the 315mb movie you sent me earlier in well under an hour, and the same video was timing out after 4 HOURS on your server. So something is slower about your server - there's no way around that.

I just don't have any suggestions for you at this time.

Sorry!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
brian
@brian
9 years ago
10,148 posts
derrickhand300:
Thinking out loud- mines converting really fast BUT like Brian was saying earlier- they are not compressed much- in fact they are the same file size as the original

So I am wondering if there is a way to let them upload initially like mine (uncompressed)

Then have something on the server compress them at an off peak time during the night?

Seems like someone would have something like this- ever heard of it?

Just wondering

Most users are not going to want to wait a day for their videos to work properly, and if you are not compressing the video in any way it could be problematic for mobile / low bandwidth users.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Yes - it's at highest which may be the issue in terms of not seeing it "pegged". Plus we did an upgrade recently which would allow it to spread out more. Another upgrade from S3 to S4 storage is imminent which will probably give another speed increase.

I'll try it on high and see what happens - thanks.


--

Ken Rich
indiegospel.net
brian
@brian
9 years ago
10,148 posts
derrickhand300:
I just did a test
Using my DSL connection I uploaded an 80 MB video in 12 minutes
Converting it ( from time I hit create to the time I could play the video) was 43 seconds
The output file is still the original 80 MB

You can see the test video quality and audio here
http://theamericandriller.com/drilling-ahead/video/280/twst
I wil lleave the link up for an hour or so then remove it

Is this using the ffmpeg settings you post above?

Thanks!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
brian
@brian
9 years ago
10,148 posts
Ken_Rich:
Yes - it's at highest which may be the issue in terms of not seeing it "pegged". Plus we did an upgrade recently which would allow it to spread out more. Another upgrade from S3 to S4 storage is imminent which will probably give another speed increase.

I'll try it on high and see what happens - thanks.

Thanks Ken - let me know if you see an improvement.

Thanks!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
I set it to high and it's still not utilizing the CPU. I put in a small MP4 video 3 minutes long, 15.4 MB, 360P from youtube.

The host sat and watched his panel and I'm registering 0-2% (see attached).

When he converts a video his goes up to over 50% for a few seconds on a small video, and it's done. I'm at over 8,000 seconds on this one so far, and still going.

Our server settings are the same, the only difference is the type of conversion script - ergo he suspects a misconfiguration in the script's (JR conversion) settings.

He said if there is something the script needs on the server, tell him where it is and he will give it, but there is nothing on his end that would prevent the CPU from working. The resources are there, it's waiting for the script to call on it, but the script isn't placing a demand.
Scree.jpg
Scree.jpg  •  94KB




--

Ken Rich
indiegospel.net

updated by @ken-rich: 02/01/15 02:49:41PM
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
I deleted the video because it just wasn't converting. A small video like that shouldn't take this long.
Video Support video_conversions 50,351 s

Something is wrong but I don't know what. I'll try it again.


--

Ken Rich
indiegospel.net

updated by @ken-rich: 02/02/15 02:38:09AM
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Tried it on highest - converted in 2 minutes. Will try again on high.

[Randy Ward]: converted 554/336/video_file from mp4 in 119.35 seconds


--

Ken Rich
indiegospel.net
Ken Rich
Ken Rich
@ken-rich
9 years ago
926 posts
Tried it on high and the same video took almost 9 minutes. However, this is the same video which wouldn't convert at all yesterday. I'm so confused - somebody shoot me...

Yesterday:
High - no convert at all - discontinued at 50,351 s

Today:
High - converted 554/337/video_file from mp4 in 526.28 seconds
Highest - converted 554/336/video_file from mp4 in 119.35 seconds

Video Specs:

Format : MPEG-4
File size : 15.5 MB
Duration : 3mn 1s
Overall bit rate mode : Variable
Overall bit rate : 716 Kbps
Confused.jpg
Confused.jpg  •  15KB




--

Ken Rich
indiegospel.net

updated by @ken-rich: 02/02/15 06:14:24AM