new server (very fast) still some issues remain

soaringeagle
@soaringeagle
6 years ago
3,304 posts
got a new server (performance test 2330 - 2460 scores)
but all audio although it converts but wont play with error
An error was encountered loading the media URL

additionaly files over a few megs fail to upload (1 gig vid)

running integrity check gets stuck after a minute

running force audio reconversion gets stuck after adding only 200 to the queue


and updating core fails as well

im assuming its a timeout issue ..but ..what timeout would it be
well the media url thing baffles me



--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities

updated by @soaringeagle: 05/05/18 08:42:52PM
soaringeagle
@soaringeagle
6 years ago
3,304 posts
may have solved all but the invalid media url issue


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
soaringeagle
@soaringeagle
6 years ago
3,304 posts
nope, closer but still getting this error in console when the audio reconversion add to queue thing stops

HTTP500: SERVER ERROR - The server encountered an unexpected condition that prevented it from fulfilling the request.
(XHR)POST - https://www.dreadlockssite.com/uploaded_audio/reconvert_save/__ajax=1


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities

updated by @soaringeagle: 02/03/18 11:35:38AM
derrickhand300
@derrickhand300
6 years ago
1,353 posts
Read 'Um And Weep :)
Apparently you get extra points bonus for multiple processors
Capture.JPG
Capture.JPG  •  83KB


updated by @derrickhand300: 02/03/18 11:59:26AM
SteveX
SteveX
@ultrajam
6 years ago
2,584 posts
Not another "Very very very fast but doesn't work at all" server, is it?

You are so unlucky with your servers!


--
¯\_(ツ)_/¯ Education, learning resources, TEL, AR/VR/MR, CC licensed content, panoramas, interactive narrative, sectional modules (like jrDocs), lunch at Uni of Bristol. Get in touch if you share my current interests or can suggest better :)
derrickhand300
@derrickhand300
6 years ago
1,353 posts
SteveX:
Not another "Very very very fast but doesn't work at all" server, is it?

You are so unlucky with your servers!

hahahaha ME? I moved to a cloud server last night- they sold me by telling me it was the closest thing you could get to dedicated without getting dedicated...Now Im on "Cloud 2" i wish i was on "Cloud 9"...beam me up scotty
Capture.JPG
Capture.JPG  •  34KB

soaringeagle
@soaringeagle
6 years ago
3,304 posts
its a setting somewhere thats the issue
php most likely
i increased max ecxec time and max input time dramaticly, it increased how long it goes before hanging up..

instead of 60 sec it goes a few minutes zat least then i get this error

i have a suspicion its some hard limit in plesk, which i was told by them was the best optiopn, i know everyone will say6 go with no control panel, but had a lot of trouble last time we tried that


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
soaringeagle
@soaringeagle
6 years ago
3,304 posts
php 7.2.1 fully compatible?


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
soaringeagle
@soaringeagle
6 years ago
3,304 posts
found clues in error log

(104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server, referer: https://www.dreadlockssite.com/uploaded_audio/reconvert

Premature end of script headers: router.php, referer: https://www.dreadlockssite.com/uploaded_audio/reconvert

mod_fcgid: HTTP request length 134225485 (so far) exceeds MaxRequestLen (134217728), referer: https://www.dreadlockssite.com/uploaded_video/create



--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
soaringeagle
@soaringeagle
6 years ago
3,304 posts
noew i get
Premature end of script headers: router.php, referer: https://www.dreadlockssite.com/uploaded_audio/reconvert

isuspect its something to do with max header size


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
soaringeagle
@soaringeagle
6 years ago
3,304 posts
pretty sure i have it solved now? i'd correct 1 error which would reveal another, and bit by biit had it narrowed down to fastcgi settings

in vhost config i added
<IfModule mod_fcgid.c>
	FcgidMaxRequestLen 64424509440
	FcgidIOTimeout 18000
	FcgidMaxRequestsPerProcess 100
	FcgidProcessLifeTime 7200
	FcgidBusyTimeout 98000
</IfModule>

i believe the busy timeouts the final piece of the puzzle as i started getting partial vid uploads


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
SteveX
SteveX
@ultrajam
6 years ago
2,584 posts
derrickhand300:hahahaha ME?
No, not you. I meant @soaringeagle

Easy mistake to make though, you do both have similar interests. But Soaring takes it to an entirely different altitude, must be said.


--
¯\_(ツ)_/¯ Education, learning resources, TEL, AR/VR/MR, CC licensed content, panoramas, interactive narrative, sectional modules (like jrDocs), lunch at Uni of Bristol. Get in touch if you share my current interests or can suggest better :)
soaringeagle
@soaringeagle
6 years ago
3,304 posts
i actualy have the issue solved
i'll be posting it as a tip in a few minhen its all verified

nobody in all these years of dealing with the same issue ever suggested it was fastcgi settings causing the issue

its a good thing i never give up.. this took step by step propblem solving till i found a working solution


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
SteveX
SteveX
@ultrajam
6 years ago
2,584 posts
Glad you've got your personal server problems sorted (for a day or two, probably).


--
¯\_(ツ)_/¯ Education, learning resources, TEL, AR/VR/MR, CC licensed content, panoramas, interactive narrative, sectional modules (like jrDocs), lunch at Uni of Bristol. Get in touch if you share my current interests or can suggest better :)
soaringeagle
@soaringeagle
6 years ago
3,304 posts
well thanks for being so helpful


or.. whatever you call that


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
SteveX
SteveX
@ultrajam
6 years ago
2,584 posts
It isn't really possible to help. Your issues are unique to your server (did you ever manage to get back in touch with the server guy who set that up for you?) and so are impossible to test against.

No matter though, you always seem to manage to get there in the end and post your serial successes here. Well done!


--
¯\_(ツ)_/¯ Education, learning resources, TEL, AR/VR/MR, CC licensed content, panoramas, interactive narrative, sectional modules (like jrDocs), lunch at Uni of Bristol. Get in touch if you share my current interests or can suggest better :)
soaringeagle
@soaringeagle
6 years ago
3,304 posts
it is absolutely NOT unique to my server especially since this is A DIFERENT SERVER
it is because my sites huge, has a massive database, and trying to allow full lenhth video uploads (multiple gigs)

this was set up by a completely diferent guy, a hosting company. it is an issue that exists in jamroom, but only when you try to use it the way i use it, so requires special server settings THAT YOU COULD HAVE TRIED TO HELP WITH
all the errors are here that i used to diagnose it WITH NO LINUX EXPERIENCE so i would ecxpect people who have done this sort of thing before might just be able to figure iot out a lil easier then me, who never touched Linux before last year.

the fast fgi settings i will post should be added to the dovcumentation fort all large sites especially those wanting to alow large file uploads

the isue is with fastcgi not my server not anyonres server
it is simply something that nobody here ever mentioned changing configuration values would solve my issues


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
soaringeagle
@soaringeagle
6 years ago
3,304 posts
argh still having 1 warning though the everything is still progressing

(70008)Partial results are valid but processing is incomplete: mod_fcgid: can't get data from http client, referer: https://www.dreadlockssite.com/uploaded_audio/reconvert

a lot closer to having this solved.
its at least still progressing beyond that point at least
its a tricky issue to solve
most agree the main influence is FcgidMaxRequestLen
i don't see any issues with tmp being constricted that could cause that
but at least for the 1st time ever integrity check completes with all options checked

this warning isn't seeming to cause any problems but it is what i have been using as diagnostics that got me this far (well the most recent tweaks) just not sure how to get rid of it completely yet...
close though..at least it seems everythings working
and i am certain this also was the cause of my backup idsssues, my vid conversion issues. every issue i had was basicly not having any direction on how to configure fastcgi


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
SteveX
SteveX
@ultrajam
6 years ago
2,584 posts
Glad that's nearly sorted then.

I can't try to help with your server settings, I doubt that the Jamroom Team can either.

Nowadays I use jamroom hosting for most of my sites (saves so much time and effort) and run a few digitalocean vps plus a hosted setup I've had for 15 years or so.

If it's a completely new server and a different server guy then why are you having the same problems? Even for very large sites it's much easier than this nowadays? Don't cut corners.


--
¯\_(ツ)_/¯ Education, learning resources, TEL, AR/VR/MR, CC licensed content, panoramas, interactive narrative, sectional modules (like jrDocs), lunch at Uni of Bristol. Get in touch if you share my current interests or can suggest better :)
soaringeagle
@soaringeagle
6 years ago
3,304 posts
im having the same problems BECAUSE NOWHERE ANYWHERE ON THIS SITE DOES IT SUGGEST CHANGING FASTCGI SETTINGS
get it?

this should be in the documentation for all large sites
jamroom hosting would have to set up a customized hopst for me because the sites so huge. and that would be 3-4 times what im paying now

i won't know if its completely soved for another houir or so but looks good so0 far

and this IS ABSOLUTELY SOMETHINMG JAMROOM SHOULD KNOW TO HELP WITH ALL SERVERS because it will affect any and all servers when the database gets this big, and when you attempt uploading larger videos for conversion.

this should be in the documentation
or at least the tips for larger servers section

cause this has been at least a year or 2 or more that its been a hassle i have been trying to get help from literally everyone

does youir server have a 2 or 3 gig database, does it have 32 gigs ram so you can upload 8 gig videos
have you tried uploading and converting vidfs over a gig

once your sites grown to a million pages with hundreds of thousands of photos, 7000 audio tracks and hundreds of videos you'll be running into the exact same issues no matter where its hosted... cause its an issue with fast cgi settings thats common to all servers (though defaults vary slightly, but not enough to matter)


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
SteveX
SteveX
@ultrajam
6 years ago
2,584 posts
Ok, I'm sure anyone with similar problems will search the forum and find your so far so good solution.

Perhaps you should mark this thread as a User Tip?


--
¯\_(ツ)_/¯ Education, learning resources, TEL, AR/VR/MR, CC licensed content, panoramas, interactive narrative, sectional modules (like jrDocs), lunch at Uni of Bristol. Get in touch if you share my current interests or can suggest better :)
soaringeagle
@soaringeagle
6 years ago
3,304 posts
no going to write a usder tip once i am certain of the solution (video upload completers, and, actually converts0 thats the final test.

i believe i am there though..
once i write the tip i hope one of the jr crew will take a look at it maybe offer a fine tuning tip, and clarify some of the explanitions

like adding a fscgi statement with a timeout for some reason got me a 1/4 of the way there and i have no idea why
i was actually expecting my site to not load after adding it but it made a big improvement which meant i was barking up the right tree


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
brian
@brian
6 years ago
10,148 posts
I've only got 2 minutes here so can't follow up much but you should NOT be using fastcgi. You need to be using Apache with mod_php OR nginx with php-fpm - both of these work really well with JR with minimal fuss.

I've built literally hundreds of servers and have not had any of the issues you are posting. If you want to go witth php-fpm I can post the nginx.conf file and www.conf file I personally use.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
brian
@brian
6 years ago
10,148 posts
Also - here's the JR system requirements page:

https://www.jamroom.net/the-jamroom-network/documentation/getting-started/774/server-requirements

It's not going to talk about fastcgi since we would never recommend it. I will get something on there that specifically tells you to NOT use it.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
soaringeagle
@soaringeagle
6 years ago
3,304 posts
hmm well i thought nginx was what causes profile domains to not work


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
brian
@brian
6 years ago
10,148 posts
Correct- if you need profile domains then use Apache with mod_php if you can. If you really need the per-user process handling of fastcgi then you'll want to play with it - it's not something we develop or test on.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
soaringeagle
@soaringeagle
6 years ago
3,304 posts
well the settings i finaly set sdeem to be working finaly but i'll look into mod_php


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities
soaringeagle
@soaringeagle
6 years ago
3,304 posts
would mod_php need any config tweaks
this is the only way i was able to get fast cgi to work
Include /var/www/vhosts/dreadlockssite.com/httpdocs/data/media/0/0/*.conf
<Proxy fcgi://127.0.0.1:9000>
	ProxySet timeout=9800
</Proxy>
<IfModule mod_fcgid.c>
	FcgidMaxRequestLen 13884901888
	FcgidIOTimeout 98000
	FcgidMaxRequestsPerProcess 100
	FcgidProcessLifeTime 7800
	FcgidBusyTimeout 98000
	FcgidIdleTimeout 600
</IfModule>



--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities

Tags