Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

MySQL crash - log files

Discussion in 'Database Discussions' started by TopSecret1, Nov 17, 2017.

Tags:
  1. TopSecret1

    TopSecret1 Registered

    Joined:
    Nov 17, 2017
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Las Vegas
    cPanel Access Level:
    Website Owner
    Hi there,

    I'm trying to figure out why our MySQL server went down last night.

    I found a lot of errors in our logs.

    Here is one of the most important errors:
    SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

    I did as Bluehost suggested and checked the error file here:
    /var/lib/mysql/username.err

    and read this thread from the past:
    VPS Down What logs file need to be checked

    Here are only some of the errors found in the log. I'm hoping you can help tell me what caused the crash..

    Code:
    2017-11-17 02:55:46 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:46 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:47 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:47 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:48 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:48 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:49 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:49 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:50 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:50 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:51 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:51 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:52 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:52 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:53 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:53 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:54 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:54 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:55 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:55 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:56 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:56 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:57 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:57 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:58 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:58 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:55:59 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:55:59 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:00 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:00 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:01 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:01 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:02 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:02 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:03 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:03 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:04 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:04 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:05 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:05 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:06 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:06 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:07 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:07 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:08 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:08 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:09 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:09 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:10 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:10 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:11 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:11 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:12 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:12 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:13 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:13 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:14 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:14 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:15 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:15 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:16 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:16 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:17 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:17 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:18 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:18 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:19 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:19 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:20 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:20 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:21 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:21 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:22 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:22 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:23 14876 [ERROR] InnoDB: Unable to lock ./ibdata1, error: 11
    2017-11-17 02:56:23 14876 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
    2017-11-17 02:56:23 14876 [Note] InnoDB: Unable to open the first data file
    2017-11-17 02:56:23 7f62dd70a720  InnoDB: Operating system error number 11 in a file operation.
    InnoDB: Error number 11 means 'Resource temporarily unavailable'.
    InnoDB: Some operating system error numbers are described at
    InnoDB: http://dev.mysql.com/doc/refman/5.6/en/operating-system-error-codes.html
    2017-11-17 02:56:23 14876 [ERROR] InnoDB: Can't open './ibdata1'
    2017-11-17 02:56:23 14876 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data!
    2017-11-17 02:56:23 14876 [ERROR] Plugin 'InnoDB' init function returned error.
    2017-11-17 02:56:23 14876 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
    2017-11-17 02:56:23 14876 [ERROR] Unknown/unsupported storage engine: InnoDB
    2017-11-17 02:56:23 14876 [ERROR] Aborting
    
    2017-11-17 02:56:23 14876 [Note] Binlog end
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'partition'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'ARCHIVE'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_FT_DELETED'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_METRICS'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_CMPMEM'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_CMP_RESET'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_CMP'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_LOCKS'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'INNODB_TRX'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'BLACKHOLE'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'MRG_MYISAM'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'CSV'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'MEMORY'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'MyISAM'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'sha256_password'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'mysql_old_password'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'mysql_native_password'
    2017-11-17 02:56:23 14876 [Note] Shutting down plugin 'binlog'
    2017-11-17 02:56:23 14876 [Note] /usr/sbin/mysqld: Shutdown complete
    
    2017-11-17 02:56:23 15208 [Note] Plugin 'FEDERATED' is disabled.
    2017-11-17 02:56:23 15208 [Note] InnoDB: Using atomics to ref count buffer pool pages
    2017-11-17 02:56:23 15208 [Note] InnoDB: The InnoDB memory heap is disabled
    2017-11-17 02:56:23 15208 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
    2017-11-17 02:56:23 15208 [Note] InnoDB: Memory barrier is not used
    2017-11-17 02:56:23 15208 [Note] InnoDB: Compressed tables use zlib 1.2.3
    2017-11-17 02:56:23 15208 [Note] InnoDB: Using Linux native AIO
    2017-11-17 02:56:23 15208 [Note] InnoDB: Using CPU crc32 instructions
    2017-11-17 02:56:23 15208 [Note] InnoDB: Initializing buffer pool, size = 128.0M
    2017-11-17 02:56:23 15208 [Note] InnoDB: Completed initialization of buffer pool
    2017-11-17 02:56:23 15208 [Note] InnoDB: Highest supported file format is Barracuda.
    2017-11-17 02:56:23 15208 [Note] InnoDB: Log scan progressed past the checkpoint lsn 23828904506
    2017-11-17 02:56:23 15208 [Note] InnoDB: Database was not shutdown normally!
    2017-11-17 02:56:23 15208 [Note] InnoDB: Starting crash recovery.
    2017-11-17 02:56:23 15208 [Note] InnoDB: Reading tablespace information from the .ibd files...
    2017-11-17 02:56:23 15208 [Note] InnoDB: Restoring possible half-written data pages
    2017-11-17 02:56:23 15208 [Note] InnoDB: from the doublewrite buffer...
    InnoDB: Doing recovery: scanned up to log sequence number 23828927657
    2017-11-17 02:56:23 15208 [Note] InnoDB: Starting an apply batch of log records to the database...
    InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
    InnoDB: Apply batch completed
    2017-11-17 02:56:24 15208 [Note] InnoDB: 128 rollback segment(s) are active.
    2017-11-17 02:56:24 15208 [Note] InnoDB: Waiting for purge to start
    2017-11-17 02:56:24 15208 [Note] InnoDB: 5.6.35 started; log sequence number 23828927657
    2017-11-17 02:56:24 15208 [Note] Server hostname (bind-address): '*'; port: 3306
    2017-11-17 02:56:24 15208 [Note] IPv6 is available.
    2017-11-17 02:56:24 15208 [Note]   - '::' resolves to '::';
    2017-11-17 02:56:24 15208 [Note] Server socket created on IP: '::'.
    2017-11-17 02:56:24 15208 [Note] Event Scheduler: Loaded 0 events
    2017-11-17 02:56:24 15208 [Note] /usr/sbin/mysqld: ready for connections.
    Version: '5.6.35'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
    2017-11-17 02:56:24 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_wrdp1/wp_options' is marked as crashed and should be repaired
    2017-11-17 02:56:24 15208 [Warning] Checking table:   './topsect2_wrdp1/wp_options'
    2017-11-17 02:57:59 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_wrdp1/wp_posts' is marked as crashed and should be repaired
    2017-11-17 02:57:59 15208 [Warning] Checking table:   './topsect2_wrdp1/wp_posts'
    2017-11-17 02:58:00 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_wrdp1/wp_postmeta' is marked as crashed and should be repaired
    2017-11-17 02:58:00 15208 [Warning] Checking table:   './topsect2_wrdp1/wp_postmeta'
    2017-11-17 03:00:39 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_wrdp1/wp_yoast_seo_meta' is marked as crashed and should be repaired
    2017-11-17 03:00:39 15208 [Warning] Checking table:   './topsect2_wrdp1/wp_yoast_seo_meta'
    2017-11-17 04:20:08 15208 [Warning] IP address '49.4.140.65' could not be resolved: Name or service not known
    2017-11-17 08:41:40 15208 [Warning] IP address '121.207.227.164' could not be resolved: Name or service not known
    2017-11-17 14:22:49 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_xc4/xcart_config' is marked as crashed and should be repaired
    2017-11-17 14:22:49 15208 [Warning] Checking table:   './topsect2_xc4/xcart_config'
    2017-11-17 14:22:49 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_xc4/xcart_form_ids' is marked as crashed and should be repaired
    2017-11-17 14:22:49 15208 [Warning] Checking table:   './topsect2_xc4/xcart_form_ids'
    2017-11-17 14:22:49 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_xc4/xcart_session_history' is marked as crashed and should be repaired
    2017-11-17 14:22:49 15208 [Warning] Checking table:   './topsect2_xc4/xcart_session_history'
    2017-11-17 14:22:50 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_xc4/xcart_session_unknown_sid' is marked as crashed and should be repaired
    2017-11-17 14:22:50 15208 [Warning] Checking table:   './topsect2_xc4/xcart_session_unknown_sid'
    2017-11-17 14:30:51 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_xc4/xcart_sessions_data' is marked as crashed and should be repaired
    2017-11-17 14:30:51 15208 [Warning] Checking table:   './topsect2_xc4/xcart_sessions_data'
    2017-11-17 14:30:52 15208 [ERROR] /usr/sbin/mysqld: Table './topsect2_wrdp1/wp_yumprint_recipe_view' is marked as crashed and should be repaired
    2017-11-17 14:30:52 15208 [Warning] Checking table:   './topsect2_wrdp1/wp_yumprint_recipe_view'
    
    
     
    #1 TopSecret1, Nov 17, 2017
    Last edited by a moderator: Nov 17, 2017
  2. rpvw

    rpvw Well-Known Member

    Joined:
    Jul 18, 2013
    Messages:
    579
    Likes Received:
    177
    Trophy Points:
    43
    Location:
    Spain
    cPanel Access Level:
    Root Administrator
  3. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    41,502
    Likes Received:
    1,616
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    The log output suggests potential InnoDB corruption. The link referenced in the previous post is a good place to start.

    Thank you.
     
Loading...

Share This Page