migration preperation incremental migration and larger sites tips

soaringeagle
@soaringeagle
11 years ago
3,304 posts
as we prepare to migrate and have uploaded some test data our sites on ning are still active and large sites are bound to have archiving as well as importing issues, so i wanted to make a quick guide on ways to make it less painful

on large ning archives you will most likely get some "unable to write file to disk" errors that stop the archive of that media files section dead and you cant restart it for 24 hours
when this happens step 1 look in the media files folder (ie photos ifyour downloading photos archive) sort by file name wich has a sequential number as a prefix
find the last photo that downloaded'

next in notepad open the local json (ie ning-photos-local.json) it will take a mommentto openj in notepad and might freeze up til its open..and wordwrap will also freeze it up a min or 2 bbut helps in the next step
you will see html code in a very unreadable format but try to find any plain english text as close to the html at the end as you can find
it doesnt matter if its something silly like dude i love your shirt wantto trade
aslong as u select enough that its unique and shouldnt have multiple results
now go to your ning search and search by that text (now that i think of it this is what im doing in discussions abnnd groups its less likely to be a proboem in photos unless its in a comment)
once you find that text block open the link (page comment whatever)
look for the last image u saw that downloaded successfully and right below it should be a broken image,, u can edit it and delete that image..or the whole comment or discussion as u see fit
wait 24 hours and start all over again

ning says they will release a fix that just skips over them..but who kbnows when that will happen the latest word isthe 3.0archiver might still take months "and they call that soon) to complete

now on uploading a large uploaded archive might take days to upload i think mine took 4 or 5 days to upload
then the import takes days because kept havintg server timeouts

if you use litespeed server on apache u have to max out the timeours set everything high as ccan be ..just fir the imoort process you will want to reduce everything around defaults when u go live

once the imports complete the media conversion can take a long time if you have thiusands of songs

now even once u finaly got a full archive your sites active daily
so do a fresh archive

when you ftp select just the json files and choose overwrite
then when you select the media files and it says exists on server overwrite skip or resule choose skip this way only the new files are uploaded
repeat the import it will seem to do nothing for a olong time as its going through the list and saying this already exists (if you turn on error reporting you will see this but it makes it alot slower)

if you do this often when you are ready to migrate and go live all you have to do is swich dns (or name servers)
and your sites live
do 1 final archive and import after you verify the dns propegated worldwide (google global dns propegation test make sure it resolves to new ip)

you want to make sure that everyone in the worlds accessing the new site before doing the final archive to be sure you colect every bit of data

then you can do the final step... say bye bye ning

this incremental approach will make the final upload and import faster and will alow even new members to test out the new site during the pre-migration stages




--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities

updated by @soaringeagle: 07/09/16 09:21:02AM
Strumelia
Strumelia
@strumelia
11 years ago
3,603 posts
Good advice Soaring about doing the incremental rather than replacing with a whole new archive. :)


--
...just another satisfied Jamroom customer.
Migrated from Ning to Jamroom June 2015

updated by @strumelia: 12/29/14 09:23:54AM
soaringeagle
@soaringeagle
11 years ago
3,304 posts
i was not looking forward to a week long migration process
and sinde ive been unsable to sleep hardly at all since begining to work on this this thought was running through my mind last night

and cause im having to redo my archive to clear out broken images in groups and discussions every 24 hours i figured might as well start doing daily imports to minimize the final impact


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities

updated by @soaringeagle: 12/29/14 09:23:54AM
Strumelia
Strumelia
@strumelia
11 years ago
3,603 posts
I find that a few minutes of playing my banjo or my Cretan Lyra help ease the stress. ;)


--
...just another satisfied Jamroom customer.
Migrated from Ning to Jamroom June 2015

updated by @strumelia: 12/29/14 09:23:54AM
soaringeagle
@soaringeagle
11 years ago
3,304 posts
im not so much stressed as excited and super busy lol overwhelmed with how much i gotta do still
but im loving it

tho theres been a few frustrating momments where i just couldnt figure things out

im mostly just exhausted from overwhelming inspiration
constant flow of new ideas i want to impliment


--
soaringeagle
head dreadhead at dreadlocks site
glider pilot student and member/volunteer coordinator with freedoms wings international soaring for people with disabilities

updated by @soaringeagle: 12/29/14 09:23:54AM
SteveX
SteveX
@ultrajam
11 years ago
2,584 posts
Strumelia:
I find that a few minutes of playing my banjo or my Cretan Lyra help ease the stress. ;)

Cool!


--
¯\_(ツ)_/¯ Education, learning resources, TEL, AR/VR/MR, CC licensed content, panoramas, interactive narrative, sectional modules (like jrDocs), lunch at Uni of Bristol. Get in touch if you share my current interests or can suggest better :)

updated by @ultrajam: 12/29/14 09:23:54AM

Tags