Mongo no longer connects

June 23, 2017 30.2k views
MongoDB Ubuntu 16.04
robynover
By:
robynover

MongoDB was running fine, but now it won’t connect. When I run mongo, I get this message:

MongoDB shell version: 3.2.12
connecting to: test
2017-06-23T01:56:34.256-0400 W NETWORK  [thread1] Failed to connect to 127.0.0.1:27017, in(checking socket for error after poll), reason: errno:111 Connection refused
2017-06-23T01:56:34.256-0400 E QUERY    [thread1] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed :
connect@src/mongo/shell/mongo.js:229:14
@(connect):1:6

exception: connect failed

I have tried restarting the mongod service, but that doesn’t help. Can you please guide me on how to debug this issue?

1 comment
  • hansen June 23, 2017

    Hi @robynover Can you post your MongoDB error log?
    Try to run tail -30 /var/log/mongodb/mongodb.log to display the last 30 lines.

Log In to Comment
2 Answers
hansen June 23, 2017

@robynover
Can you delete this file and then launch mongo service: /tmp/mongodb-27017.sock

When you start mongo directly it starts with default options. If you look at your first error logs, you’ll see it’s set to /var/lib/mongodb

Reply
  • robynover June 24, 2017

    Thank you @hansen – deleting that file worked! If you’re up for it, I’d appreciate any additional info about why this might have happened or why this fix works. Thanks!

    • hansen June 24, 2017

      Can you post the entire /var/log/mongodb/mongod.log to www.pastebin.com and link it here.
      It usually happens when mongo crashes (or the entire server is hard rebooted), but without seeing all the logs, it’s hard to tell.
      But even with the logs, we might not find the solution - it depends on how well the logging was around the crash time.

      • robynover June 24, 2017
        • hansen June 24, 2017

          The server ran out of available RAM at least once (2017-06-08T04:24:23.444+0000) - and it also looks like it had an unclean shutdown (crash or hard reboot) at least once (2017-06-08T10:55:16.809+0000)

          I know those dates are not today, but if something like that has happened before, then I would guess it could be the cause of today’s problems.
          To dig much deeper, we would also have to look at the syslog (and probably other logs too) going back the same period - but most setups don’t have logging going back that far unless specifically configured.

robynover June 23, 2017

Thank you @hansen. I don’t have a mongodb.log but I do have mongod.log:

2017-06-23T01:48:28.663-0400 I CONTROL  [initandlisten]     distmod: ubuntu1604
2017-06-23T01:48:28.663-0400 I CONTROL  [initandlisten]     distarch: x86_64
2017-06-23T01:48:28.663-0400 I CONTROL  [initandlisten]     target_arch: x86_64
2017-06-23T01:48:28.663-0400 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, storage: { dbPath: "/var/lib/mongodb", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log", quiet: true } }
2017-06-23T01:48:28.720-0400 E NETWORK  [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock errno:1 Operation not permitted
2017-06-23T01:48:28.720-0400 I -        [initandlisten] Fatal Assertion 28578
2017-06-23T01:48:28.720-0400 I -        [initandlisten] 

***aborting after fassert() failure


2017-06-23T01:56:30.912-0400 I CONTROL  [main] ***** SERVER RESTARTED *****
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten] MongoDB starting : pid=19105 port=27017 dbpath=/var/lib/mongodb 64-bit host=currency
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten] db version v3.2.12
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten] git version: ef3e1bc78e997f0d9f22f45aeb1d8e3b6ac14a14
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten] allocator: tcmalloc
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten] modules: none
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten] build environment:
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten]     distmod: ubuntu1604
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten]     distarch: x86_64
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten]     target_arch: x86_64
2017-06-23T01:56:30.930-0400 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, storage: { dbPath: "/var/lib/mongodb", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log", quiet: true } }
2017-06-23T01:56:30.976-0400 E NETWORK  [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock errno:1 Operation not permitted
2017-06-23T01:56:30.976-0400 I -        [initandlisten] Fatal Assertion 28578
2017-06-23T01:56:30.976-0400 I -        [initandlisten] 

***aborting after fassert() failure
Reply
  • robynover June 23, 2017

    When I run mongod, this is what I get:

    2017-06-23T12:32:00.096-0400 I CONTROL  [initandlisten] MongoDB starting : pid=18915 port=27017 dbpath=/data/db 64-bit host=currency
2017-06-23T12:32:00.097-0400 I CONTROL  [initandlisten] db version v3.2.12
2017-06-23T12:32:00.097-0400 I CONTROL  [initandlisten] git version: ef3e1bc78e997f0d9f22f45aeb1d8e3b6ac14a14
2017-06-23T12:32:00.097-0400 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2017-06-23T12:32:00.097-0400 I CONTROL  [initandlisten] allocator: tcmalloc
2017-06-23T12:32:00.097-0400 I CONTROL  [initandlisten] modules: none
2017-06-23T12:32:00.098-0400 I CONTROL  [initandlisten] build environment:
2017-06-23T12:32:00.098-0400 I CONTROL  [initandlisten]     distmod: ubuntu1604
2017-06-23T12:32:00.098-0400 I CONTROL  [initandlisten]     distarch: x86_64
2017-06-23T12:32:00.098-0400 I CONTROL  [initandlisten]     target_arch: x86_64
2017-06-23T12:32:00.098-0400 I CONTROL  [initandlisten] options: {}
2017-06-23T12:32:00.154-0400 I STORAGE  [initandlisten] exception in initAndListen: 29 Data directory /data/db not found., terminating
2017-06-23T12:32:00.155-0400 I CONTROL  [initandlisten] dbexit:  rc: 100
    • robynover June 23, 2017

      I see that it says “Data directory /data/db not found”, and indeed there is no such directory. However, it was working fine yesterday, so I don’t understand what could have happened. Could my data directory have been deleted remotely somehow?

Have another answer? Share your knowledge.

Related