Different Redis behaviour between macOS and Ubuntu 18.04.

Question

I’m not sure where to start and where I should go to ask for the issue I’m having. Hopefully I can get solution or direction here. Apologise if these are confusing. Questions at the end.

BASE

rails 5.2 + resque + resque-schedule
enqueue job through rails resque:scheduler
has following job

	class ThisJob < ActiveJob::Base
		queue_as :high

		def perform
			cache-A = Rails.cache.fetch('cache-key', namespace: 'appname:cache') { -- cache-data -- }

			task-A # insert into database by mixing values from cache-A
		end
	end

macOS: Install redis 5 via homebrew

Ubuntu 18.04: Install redis 5 by following How To Install Redis from Source on Ubuntu 18.04

EXPECTED BEHAVIOUR

cache-key can be fetched from ApplicationController and ActionJob cache-key is stored in Redis

macOS:

Run redis-server, rails s, rails resque:scheduler, rails resque:work queue=* in that sequence from terminal.
worker executes ThisJob and it works as expected
redis-cli keys * return cache-key.

This is what i get from redis-cli monitor if cache-key is not present.

1566012687.511574 [0 127.0.0.1:58933] "smembers" "resque:workers"
1566012687.512668 [0 127.0.0.1:58933] "mget" "resque:worker:--host info--:*"
1566012687.513653 [0 127.0.0.1:58933] "smembers" "resque:workers"
1566012691.258800 [0 127.0.0.1:57822] "smembers" "resque:queues"
1566012691.259296 [0 127.0.0.1:57822] "lpop" "resque:queue:high"
1566012691.260567 [0 127.0.0.1:57822] "set" "resque:worker:--host info--:*" "{--job here--}"
1566012691.269345 [0 127.0.0.1:58976] "auth" "foobared"
1566012691.506790 [0 127.0.0.1:58977] "auth" "foobared"
1566012691.506960 [0 127.0.0.1:58977] "get" "--appname:cache:cache-key--"
1566012691.651050 [0 127.0.0.1:58977] "set" "--appname:cache:cache-key--" "\x04\bo: ActiveSupport::Cache::Entry\t:\x0b@valueI"
1566012693.568291 [0 127.0.0.1:57822] "del" "resque:worker:--host info--:*"

Ubuntu 18.04:

Boot redis-server, rails s, rails resque:scheduler, rails resque:work queue=* from systemd
worker executes ThisJob, it goes straight to cache-data, then executes task-A. It does not fetch cache-key.
redis-cli keys * does not return cache-key.

This is what i get from redis-cli monitor if cache-key is not present.

1566013554.022762 [0 127.0.0.1:60584] "smembers" "resque:workers"
1566013554.023089 [0 127.0.0.1:60584] "mget" "resque:worker:--host info--:*"
1566013554.023519 [0 127.0.0.1:60584] "smembers" "resque:workers"
1566013554.890333 [0 127.0.0.1:60574] "evalsha" "8f4a4e422bfa8f9a0931e350d6e30b1c3ce97c33" "1" "resque:resque:resque_scheduler_master_lock" "--host info--"
1566013554.890381 [0 lua] "SETNX" "resque:resque:resque_scheduler_master_lock" "--host info--"
1566013554.891184 [0 127.0.0.1:60574] "evalsha" "8d4e4994f7049799013e71b43d439219209e9f57" "1" "resque:resque:resque_scheduler_master_lock" "--host info--"
1566013554.891208 [0 lua] "GET" "resque:resque:resque_scheduler_master_lock"
1566013554.891226 [0 lua] "EXPIRE" "resque:resque:resque_scheduler_master_lock" "180"
1566013554.891236 [0 lua] "GET" "resque:resque:resque_scheduler_master_lock"
1566013554.891851 [0 127.0.0.1:60574] "zrangebyscore" "resque:delayed_queue_schedule" "-inf" "1566013554" "LIMIT" "0" "1"
1566013556.780238 [0 127.0.0.1:60580] "lpop" "resque:queue:high"
1566013556.781916 [0 127.0.0.1:60580] "set" "resque:worker:--host info--:*" "{--job here--}"
1566013556.788373 [0 127.0.0.1:60662] "auth" "--password--"
1566013558.503842 [0 127.0.0.1:60580] "del" "resque:worker:--host info--:*"

ATTEMPT in Ubuntu 18.04:

Tried to trigger ThisJob.perform_now from index.html.erb, and it works as expected.
Tried running rails s, rails resque:scheduler, rails resque:work queue=* in that sequence from terminal, and the result just like when running from systemd.
redis-cli keys * has cache-key

cache-A = Rails.cache.read('cache-key', namespace: 'appname:cache')

cache-A returns null

Questions

Is it because macOS and Ubuntu have differences in running Redis?
Is it because homebrew and apt install have differences in building Redis?
Any idea how to make this scheduler fetch Redis cache?
Something I missed?

Thank you.

Bobby · Answer

Hi there,

What I personally do in such cases, is to install Redis on a Droplet with the same OS as my production server, that way I can use the Redis instance from my local laptop and be sure that it has the exact same behavior.

An alternative option is to use Docker instead, eg:

docker run --name some-redis -d redis

Regarding your questions, there are a few things that I could suggest checking:

Redis Configuration: Check whether the Redis configurations are identical on both macOS and Ubuntu. Although Redis comes with a default configuration that should work out of the box for most cases, there might be differences due to how the Redis packages are built for different OSes or how you’ve manually configured them. The Redis configuration file is usually located at /etc/redis/redis.conf on Ubuntu.
Permissions: Ensure that the user running your Rails application and Resque workers has sufficient permissions to access Redis.
Redis Namespace: In your code, you’re using a Redis namespace ‘appname:cache’ for fetching the cache. Make sure that you’re writing to the same namespace.
Systemd Services: When running the services from Systemd, make sure the environment is correctly set. Sometimes, Systemd services don’t have access to the same environment variables as your user, which can cause issues.
Networking: Verify that there are no networking issues preventing your application from connecting to Redis. You can test this by trying to connect to your Redis server using redis-cli from the command line.
Rails Cache Configuration: Ensure that Rails is properly configured to use Redis as the cache store. You might have different configurations for different environments (development, production, etc.). Verify that the settings are correct for the environment you’re running on Ubuntu.

Regarding your specific questions:

Differences between macOS and Ubuntu: Although there are differences between the two operating systems, Redis is designed to work consistently across different systems. So, the issue is likely not due to the differences between macOS and Ubuntu per se, but rather some differences in the specific configurations on your two systems.
Differences between Homebrew and apt: Homebrew and apt are just package managers and should not cause differences in the behavior of the installed software. However, different packages may come with different default configurations, or you may have manually configured them differently.

In order to further debug the issue, you might want to look at the logs of your Rails application, Resque workers, and Redis server. These logs could provide more insights into what’s going wrong.

Best,

Bobby