Gedcom import

Hi there! I'm trying out Genosis, and it looks pretty nice! I'm having a problem though. I'm trying to import a Gedcom file, but I can't seem to finish the process. It's got just over 41000 ancestors, and I'm trying and re-trying to import the file over and over again. But it stops at one place or another. Sometimes it gets as far as to 18000, sometimes 29000, and once as far as 38000, which is as high as I got so far. And it simply stops in the middle of the import, no error or anything, just hangs there.

If I refresh, I'm back to the import page and can start again.

Do you have any idea what I could do to import the whole file?

Thanks

updated by @adrianbara: 09/06/19 09:07:24PM

Welcome to the forum!

It sounds like your server is running out of memory or processor time - are you on your own server or are you on shared hosting some where? When doing a GEDCOM import, the Genosis core module has to build a large in-memory structure that it works from - so it sounds like it is making it that far, and starts creating the necessary entries. However, there are A LOT of database queries that are going to be kicked off during that process and you could be running into a resource limitation of your hosting account (i.e X number of DB queries within X seconds of each other, etc.).

Who are you hosting with?

Thanks!


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

It's a hosting service in Sweden, called Oderland. So I could perhaps increase the php memory on the server (I have CPanel access)?

It doesn't sound like a memory issue to me - if it was, I don't think you'd even get past the parsing part and you would never see it creating the profiles. Since it stops at random places as well, and not at the same place every time, that tells me the GEDCOM file is correct and it's not hitting something it can't parse.

One thing to do is look in your jamroom's data/logs directory for an error_log - if you see that the last entries in that file can give us an idea of what might be going on.

Let me know if that helps.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net
updated by @brian: 05/27/19 07:59:12AM

I'll take a look. Thanks. Here's a list of the current settings I'm using for php 7.2.


allow_url_fopen On
always_populate_raw_post_data 0
disable_functions no value
display_errors On
error_reporting E_ALL & ~E_NOTICE & ~E_STRICT & ~E_DEPRECATED
file_uploads On
include_path .:
log_errors Off
mail.force_extra_parameters no value
max_execution_time 4000
max_file_uploads 20
max_input_time 90
max_input_vars 3000
memcached.use_sasl Off
memory_limit 2048M
newrelic.appname PHP Application
newrelic.license no value
newrelic.logfile /tmp/newrelic.log
newrelic.loglevel error
opcache.fast_shutdown Off
opcache.interned_strings_buffer 8
opcache.max_accelerated_files 2000
opcache.memory_consumption 196
opcache.revalidate_freq 2
opcache.validate_timestamps On
open_basedir no value
post_max_size 50M
register_argc_argv Off
register_long_arrays Off
session.save_handler files
session.save_path
short_open_tag On
upload_max_filesize 50M
upload_tmp_dir

I managed to fish this out from a logfile.

"Query Error: Lost connection to MySQL server during query:"

This is the root cause here - your MySQL server is crashing. It's probably running out of memory or connections, and is crashing. It shouldn't crash, but I suspect the MySQL server is misconfigured. You'll want to contact your hosting provider and have them check out the MySQL error log - it will have more info in it that can hopefully help them resolve the issue.

Let me know if that helps.


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

And now I'm getting this:

"The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later." instead of the site. Any idea what could have triggered this?

You want to contact your hosting provider - your server is underpowered for what you want to do. This is not a Jamroom error but an error from your server telling you it was unable to process your request "due to maintenance downtime or capacity problems".


--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

Thanks a lot Brian for all your replies. It was the hosting provider, just like you said. I managed to call them and have them increase the time-out value on the connection to the MySQL server. So the first part of the import imported all 41000 ancestors in my GEDCOM file. But then it started on sources, and then it hangs again. So it seems my time-out setting needs to be much much higher. Do you have any idea how much this setting should be set to? We set it to 25 minutes, but that wasn't enough. Do I need to increase it to an hour or more?

Hello again Brian, do you have any idea how many parts there is in the import process? First import, then something about sources, what exactly does it do then? Why does it need a connection to the database after it has imported the number of ancestors from the GEDCOM file?

The import process takes the GEDCOM file and puts the data contained into the sites database under an assigned tree.

Once it has finished you will see "finished importing GEDCOM file"

Then "The GEDCOM file was successfully imported".

After that the database connection is no longer needed for importing but will still be needed to view any of the information that is now in the database.