Question

Rails app getting a ActiveRecord::ConnectionTimeoutError timeout error

Posted October 16, 2014 27.7k views

Hi,

I have my website running on RoR 4.0.2 and have a connection pool of capacity as 10 for my production server. On and off I get this weird error of timeout ActiveRecord::ConnectionTimeoutError on my rails app and the website hangs. I have to restart my server to get everything back to normal.

I am running unicorn to fork the Rails server.

Any help will be really appreciated.

Error:

ActiveRecord::ConnectionTimeoutError (could not obtain a database connection within 5.000 seconds (waited 5.000 seconds)):
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:190:in `block in wait_poll'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:181:in `loop'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:181:in `wait_poll'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:136:in `block in poll'
  /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:146:in `synchronize'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:134:in `poll'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:423:in `acquire_connection'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:356:in `block in checkout'
  /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:355:in `checkout'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:265:in `block in connection'
  /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:264:in `connection'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:546:in `retrieve_connection'
  activerecord (4.0.2) lib/active_record/connection_handling.rb:79:in `retrieve_connection'
  activerecord (4.0.2) lib/active_record/connection_handling.rb:53:in `connection'
  activerecord (4.0.2) lib/active_record/query_cache.rb:51:in `restore_query_cache_settings'
  activerecord (4.0.2) lib/active_record/query_cache.rb:43:in `rescue in call'
  activerecord (4.0.2) lib/active_record/query_cache.rb:32:in `call'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:626:in `call'
  actionpack (4.0.2) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
  activesupport (4.0.2) lib/active_support/callbacks.rb:373:in `_run__2161530995940529175__call__callbacks'
  activesupport (4.0.2) lib/active_support/callbacks.rb:80:in `run_callbacks'
  actionpack (4.0.2) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  actionpack (4.0.2) lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
  actionpack (4.0.2) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
  actionpack (4.0.2) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
  railties (4.0.2) lib/rails/rack/logger.rb:38:in `call_app'
  railties (4.0.2) lib/rails/rack/logger.rb:20:in `block in call'
  activesupport (4.0.2) lib/active_support/tagged_logging.rb:67:in `block in tagged'
  activesupport (4.0.2) lib/active_support/tagged_logging.rb:25:in `tagged'
  activesupport (4.0.2) lib/active_support/tagged_logging.rb:67:in `tagged'
  railties (4.0.2) lib/rails/rack/logger.rb:20:in `call'
  actionpack (4.0.2) lib/action_dispatch/middleware/request_id.rb:21:in `call'
  rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
  rack (1.5.2) lib/rack/runtime.rb:17:in `call'
  actionpack (4.0.2) lib/action_dispatch/middleware/static.rb:64:in `call'
  rack (1.5.2) lib/rack/sendfile.rb:112:in `call'
  railties (4.0.2) lib/rails/engine.rb:511:in `call'
  railties (4.0.2) lib/rails/application.rb:97:in `call'
  activerecord (4.0.2) lib/active_record/connection_adapters/abstract/connection_pool.rb:626:in `call'
  unicorn (4.7.0) lib/unicorn/http_server.rb:580:in `process_client'
  unicorn (4.7.0) lib/unicorn/http_server.rb:660:in `worker_loop'
  unicorn (4.7.0) lib/unicorn/http_server.rb:527:in `spawn_missing_workers'
  unicorn (4.7.0) lib/unicorn/http_server.rb:153:in `start'
  unicorn (4.7.0) bin/unicorn:126:in `<top (required)>'
  /usr/local/rvm/gems/ruby-2.0.0-p353/bin/unicorn:23:in `load'
  /usr/local/rvm/gems/ruby-2.0.0-p353/bin/unicorn:23:in `<main>'
  /usr/local/rvm/gems/ruby-2.0.0-p353/bin/ruby_executable_hooks:15:in `eval'
  /usr/local/rvm/gems/ruby-2.0.0-p353/bin/ruby_executable_hooks:15:in `<main>'
Add a comment
syedsanahassan
By:
syedsanahassan

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
3 answers
asb October 16, 2014

The ActiveRecord::ConnectionTimeoutError error usually means that the database can not can’t accept any new connection connections as it is maxed out. As you are using Unicorn, you need to consider the number of connections that each forked worker process will attempt to make. If you have four Unicorn workers processes, they all may be trying to make 10 connections each.

If you post your database.yml and Unicorn configuration, we might be able to help further.

Reply Report
syedsanahassan December 19, 2014

@asb Thanks for the reply I did read it, but the post on https://bibwild.wordpress.com/2014/07/17/activerecord-concurrency-in-rails4-avoid-leaked-connections/ was really helpful.

For viewers who come here from Google search (coz DO posts are high in content value :P), I was using Thread.new in many locations where I was trying to archive data that was flowing out and therefore I opened a lot of connections and never used the rails directive with_connection when trying to make a db connection and therefore AR didn’t know how to check back the connection into the pool and I exhausted the pool with my n00bness.

Reply Report
codeminator February 16, 2015

Are you using Threads in your app dealing with the database? if so, you will have to wrap the Thread code like this

begin
#your code here
ensure
#guarantee that the thread is releasing the DB connection after it is done
ActiveRecord::Base.connectionpool.releaseconnection
end

Reply Report
Submit an Answer

Looking for something else?

Ask a new question Search for more help