Related

Join the DigitalOcean Community Community

Join 1M+ other developers and:

  • Get help and share knowledge in Q&A
  • Subscribe to topics of interest
  • Get courses & tools that help you grow as a developer or small business owner
 Join Now
Why are snapshots so slow Question Question
Is both HTTP and HTTPS versions should be verified in Webmaster Tools? Question Question

Question

Intermittent 500 error when inserting into MySQL from Python Flask App

Posted August 5, 2019 3.5k views
NginxCentOSMySQLDigitalOceanPythonDjangoPython Frameworks

I keep getting a 500 issue when inserting data into MySql DB. I know when the issue occurs but can’t figure out why.

When I insert data (e.g. new user) into my DB, I can see it has been inserted correctly from MySql Workbench but about 50% of the time a 500 error is returned which says that the entry doesn’t exist. When you refresh the page it randomly comes and goes.

After I restart uwsgi (service uwsgi restart) and use the app again, it works as expected until I add some more data then the issue occurs again but only for data inserted after the restart.

The app works as expected when running on my local windows machine.

Would really appreciate any help on this before my head explodes!

Add a comment
ollieb
By:
ollieb

Related

Join the DigitalOcean Community Community

Join 1M+ other developers and:

  • Get help and share knowledge in Q&A
  • Subscribe to topics of interest
  • Get courses & tools that help you grow as a developer or small business owner
 Join Now
Why are snapshots so slow Question Question
Is both HTTP and HTTPS versions should be verified in Webmaster Tools? Question Question

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
1 answer
KFSys August 5, 2019

Hi ollieb,

can you provide us with the mysql error log. The error should be logged and it can help us troubleshoot the issue. The error log itself and it’s path should be configured in your /etc/my.cnf file

Other than that, I would suggest your memory is running out.

To confirm that, you can check your /var/log/messages and see if there are any kill and oom signals.

Reply Report
  • ollieb August 5, 2019

    Hi Kdimitrov,

    /var/log/messages returns:
    Aug 5 18:57:20 online-car-history OCH: return self.viewfunctions[rule.endpoint](**req.view_args)
    Aug 5 18:57:20 online-car-history OCH: File “./src/models/users/decorators.py”, line 12, in decorated    function
    Aug 5 18:57:20 online-car-history OCH: return f(args, *kwargs)
    Aug 5 18:57:20 online-car-history OCH: File “./src/models/users/views.py”, line 75, in usercars
    Aug 5 18:57:20 online-car-history OCH: user = User.get    byemail(sessionemail)
    Aug 5 18:57:20 online-car-history OCH: File “./src/models/users/user.py”, line 65, in getbyemail
    Aug 5 18:57:20 online-car-history OCH: return cls(*Database.find_one(UserConstants.userTABLE, “email = ’” + email + “’”))
    Aug 5 18:57:20 online-car-history OCH: TypeError: type object argument after *     must be a mapping, not NoneType

    /var/log/mysqld.log returns:
    2019-08-04T00:10:29.231668Z 17 [Warning] [MY-010055] [Server] IP address ‘185.156.177.20’ could not be resolved: Name or service not known
    2019-08-04T03:33:30.647288Z 19 [Warning] [MY-010055] [Server] IP address '66.79.179.208’ could not be resolved: Name or service not known
    2019-08-04T05:42:53.124433Z 20 [Warning] [MY-010055] [Server] IP address '46.161.27.77’ could not be resolved: Name or service not known
    2019-08-04T15:51:05.203893Z 21 [Warning] [MY-010058] [Server] Hostname 'c180bb.rederatural.com’ does not resolve to '104.140.188.54’.
    2019-08-04T18:12:03.112902Z 22 [Warning] [MY-010055] [Server] IP address '125.64.94.211’ could not be resolved: Name or service not known
    2019-08-05T01:31:20.407156Z 23 [Warning] [MY-010057] [Server] IP address '35.189.238.222’ has been resolved to the host name '222.238.189.35.bc.googleusercontent.com’, which resembles IPv4-address itself.
    2019-08-05T07:23:35.151185Z 32 [Warning] [MY-010055] [Server] IP address '218.61.16.144’ could not be resolved: Name or service not known
    2019-08-05T08:12:28.086986Z 54 [Warning] [MY-010055] [Server] IP address '116.255.198.218’ could not be resolved: Name or service not known
    2019-08-05T16:43:05.220446Z 55 [Warning] [MY-010055] [Server] IP address '170.130.187.34’ could not be resolved: Name or service not known
    2019-08-05T17:27:48.437593Z 62 [Warning] [MY-010056] [Server] Host name 'zg-0403-47.stretchoid.com’ could not be resolved: Name or service not known

    /var/run/mysqld.pid returns:
    21249

    How do I check the memory usage?

    Thanks,
    Oli

    Reply Report
    • KFSys August 5, 2019

      Hi Oli,

      I can’t see anything causing this error in the log. Regarding the /var/log/messages log, now that I’ve seen the log you provided, I don’t believe the droplet is actually running out of memory.

      Okay, let’s try a few more things.

      Can you please check your database.py file and make sure the following is not the case

      MySQLdb.connect(host="127.0.0.1:3306", ...)

      is wrong, it should be be

      MySQLdb.connect(host="127.0.0.1", port=3306, ...)

      Hostname and port need to be supplied separately.

      Also, these are the default values, you can also omit them. Only if you use a different port/host you have to supply them.

      This suggestion is less likely to be the cause however it’s good to check it out.

      To check further logs, can you provide me the contents of the logs for flask app located at:

      /var/log/daemon.log

      Nginx logs would be welcomed as well 

      /var/log/nginx/error.log
      Reply Report
      • ollieb August 6, 2019

        Hi Kdimitrov,

        The database.py file is setup correctly and I have tried with 3 different MySql Db’s and still the same issue, please see below:

        Database.mydb = mysql.connector.connect(user =os.environ.get(‘DBUSER’), password= os.environ.get('DBUSERPASSWORD’), port =os.environ.get('DBPORT’), host = os.environ.get('DBHOST’), database=os.environ.get('DBNAME’), authplugin='mysqlnative_password’)

        There is no daemon.log file and the nginx/error.log file is empty.

        Reply Report
        • KFSys August 6, 2019

          Hi Oli,

          I see, thank you for confirming.

          In order to resolve the issue, you’ll need to start logging your errors. You can setup the error_log directly from your application.

          In order to setup your error_log you can follow this tutorial - How to write Error Log in Python Flask Web Application

          Once you do, please recreate the 500 internal error and provide the error_log’s output here.

          Reply Report
          • ollieb August 7, 2019

            Hi Kdimitrov,

            I have attached app logs and I’ve been writing to the console to work out where its failing.

            To recreate, I made testuser@test.com which is correct when getby_email function is called.

            Basically when the query asks the database for a user with that email, its randomly returns ‘None’ even though I can see on MySql workbench that the user is in the database.

            Then when the page is refreshed, it will return with the user. When I reset the uwsgi, then when I call for testuser@test.com it returns correctly 100% of the time as shown in the second testuser@test.com @ getbyemail

            Aug 07 08:09:19 online-car-history OCH[3154]: [pid: 3156|app: 0|req: 29/49] 127.0.0.1 () {50 vars in 989 bytes} [Wed Aug 7 08:09:19 2019] GET /users/cars => generated
            Aug 07 08:09:19 online-car-history OCH[3154]: {’id’: 'd43344a55f7140c89a184aff9a77107d’, 'name’: 'asdsad’, 'surname’: 'adasd’, 'email’: 'testuser@test.com’, 'password
            Aug 07 08:09:19 online-car-history OCH[3154]: testuser@test.com @ getbyemail
            Aug 07 08:09:10 online-car-history OCH[3154]: [pid: 3156|app: 0|req: 28/48] 127.0.0.1 () {38 vars in 555 bytes} [Wed Aug 7 08:09:10 2019] GET / => generated 5177 bytes
            Aug 07 08:04:52 online-car-history OCH[3154]: [pid: 3156|app: 0|req: 27/47] 127.0.0.1 () {38 vars in 533 bytes} [Wed Aug 7 08:04:52 2019] GET / => generated 5177 bytes
            Aug 07 08:04:37 online-car-history OCH[3154]: [pid: 3156|app: 0|req: 26/46] 127.0.0.1 () {40 vars in 601 bytes} [Wed Aug 7 08:04:37 2019] GET / => generated 5177 bytes
            Aug 07 08:03:49 online-car-history OCH[3154]: [pid: 3158|app: 0|req: 15/45] 127.0.0.1 () {50 vars in 989 bytes} [Wed Aug 7 08:03:49 2019] GET /users/cars => generated
            Aug 07 08:03:49 online-car-history OCH[3154]: TypeError: type object argument after ** must be a mapping, not NoneType
            Aug 07 08:03:49 online-car-history OCH[3154]: return cls(**Database.find            one(UserConstants.userTABLE, “email = ’” + email + “’”))
            Aug 07 08:03:49 online-car-history OCH[3154]: File “./src/models/users/user.py”, line 75, in getbyemail
            Aug 07 08:03:49 online-car-history OCH[3154]: user = User.getbyemail(sessionemail)
            Aug 07 08:03:49 online-car-history OCH[3154]: File “./src/models/users/views.py”, line 75, in user            cars
            Aug 07 08:03:49 online-car-history OCH[3154]: return f(args, *kwargs)
            Aug 07 08:03:49 online-car-history OCH[3154]: File “./src/models/users/decorators.py”, line 12, in decoratedfunction
            Aug 07 08:03:49 online-car-history OCH[3154]: return self.view            functions[rule.endpoint](**req.viewargs)
            Aug 07 08:03:49 online-car-history OCH[3154]: File “/usr/local/lib/python3.7/site-packages/flask/app.py”, line 1818, in dispatch            request
            Aug 07 08:03:49 online-car-history OCH[3154]: rv = self.dispatchrequest()
            Aug 07 08:03:49 online-car-history OCH[3154]: File “/usr/local/lib/python3.7/site-packages/flask/app.py”, line 1832, in full            dispatchrequest
            Aug 07 08:03:49 online-car-history OCH[3154]: raise value
            Aug 07 08:03:49 online-car-history OCH[3154]: File “/usr/local/lib/python3.7/site-packages/flask/            compat.py”, line 36, in reraise
            Aug 07 08:03:49 online-car-history OCH[3154]: reraise(exctype, excvalue, tb)
            Aug 07 08:03:49 online-car-history OCH[3154]: File “/usr/local/lib/python3.7/site-packages/flask/app.py”, line 1737, in handleuserexception
            Aug 07 08:03:49 online-car-history OCH[3154]: rv = self.handleuserexception(e)
            Aug 07 08:03:49 online-car-history OCH[3154]: File “/usr/local/lib/python3.7/site-packages/flask/app.py”, line 1834, in fulldispatchrequest
            Aug 07 08:03:49 online-car-history OCH[3154]: response = self.fulldispatchrequest()
            Aug 07 08:03:49 online-car-history OCH[3154]: File “/usr/local/lib/python3.7/site-packages/flask/app.py”, line 2311, in wsgiapp
            Aug 07 08:03:49 online-car-history OCH[3154]: Traceback (most recent call last):
            Aug 07 08:03:49 online-car-history OCH[3154]: [2019-08-07 08:03:49,655] ERROR in app: Exception on /users/cars [GET]
            Aug 07 08:03:49 online-car-history OCH[3154]: None User does NOT exist
            Aug 07 08:03:49 online-car-history OCH[3154]: test            user@test.com @ getbyemail

            Reply Report

Related

Looking for something else?

Ask a new question Search for more help