Related

Product App Platform: Run Rails apps without managing servers Product
PHP Error Log: authz_core:error - For files that don't exists on my server Question Question
blocked ip in spam Question Question

Question

Dokku Rails app unable to interact with Block Storage [Errno::EACCES (Permission denied)]

Posted May 8, 2017 5.7k views
Ruby on RailsDokkuUbuntu 16.04Block Storage

I have a rails app where users are able to upload an avatar image via Ruby’s Paperclip gem.

In terms of paperclip’s configurations, I have the following set:

For my model:
hasattachedfile :avatar, path: “/mnt/volume-sgp1-01/:class/:attachment/:id/:style/:filename”

For paperclip environment configurations:
config.paperclip_defaults = { url: ’/mnt/volume-sgp1-01/:class/:attachment/:id/:style/:filename’ }

Hypothetically, this should work. However, when the upload starts, I get the following error message.

Errno::EACCES (Permission denied @ dirsmkdir - /mnt/volume-sgp1-01)

Also, once permission is granted, how can I make sure that it is automatically linked to a corresponding url direction in my rails app? (ex: xyz.com/mnt/volume-sgp1-01/…)

Thanks!

I have read the Digital Ocean documentation on block storage, but I feel the documentation is missing a lot of important details on actually integrating it with a Ruby on Rails application.

Add a comment
sergiorivasf40c7ff6ddb06ce
By:
sergiorivasf40c7ff6ddb06ce

Related

Product App Platform: Run Rails apps without managing servers Product
PHP Error Log: authz_core:error - For files that don't exists on my server Question Question
blocked ip in spam Question Question

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
3 answers
sergiorivasf40c7ff6ddb06ce May 20, 2017

After posting up a similar question on Stack Overflow, I was guided by a dokku user to this plugin which is native to Dokku and should solve all these problems.

Documentation about using Persistent Storage on Dokku

Reply Report
hansen May 8, 2017

Hi @sergiorivasf40c7ff6ddb06ce

I don’t know anything about Dokku Rails, but the Block Storage works just like any other mount point (like another disk connected to the server), so you shouldn’t have to do anything other that making sure the directory is chown by the correct user/group and have chmod to a setting allowing for read and write.

Reply Report
  • sergiorivasf40c7ff6ddb06ce May 10, 2017

    Hey, I’m completely unfamiliar with both chown and chmod, as prior to this I did all my app development locally on my Mac, which by default never had problems with permissions.
    Could you tell me what I should be typing in the command line?

    In terms of user accounts, this is what I have available:

    Normal User Accounts:
    dokku:x:1000:1000:,,,:/home/dokku:/bin/bash

    System User Accounts:
    root:x:0:0:root:/root:/bin/bash
    daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
    bin:x:2:2:bin:/bin:/usr/sbin/nologin
    sys:x:3:3:sys:/dev:/usr/sbin/nologin
    sync:x:4:65534:sync:/bin:/bin/sync
    games:x:5:60:games:/usr/games:/usr/sbin/nologin
    man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
    lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
    mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
    news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
    uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
    proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
    www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
    backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
    list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
    irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin
    gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
    nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
    systemd-timesync:x:100:102:systemd Time Synchronization,,,:/run/systemd:/bin/false
    systemd-network:x:101:103:systemd Network Management,,,:/run/systemd/netif:/bin/false
    systemd-resolve:x:102:104:systemd Resolver,,,:/run/systemd/resolve:/bin/false
    systemd-bus-proxy:x:103:105:systemd Bus Proxy,,,:/run/systemd:/bin/false
    syslog:x:104:108::/home/syslog:/bin/false
    _apt:x:105:65534::/nonexistent:/bin/false
    lxd:x:106:65534::/var/lib/lxd/:/bin/false
    messagebus:x:107:111::/var/run/dbus:/bin/false
    uuidd:x:108:112::/run/uuidd:/bin/false
    dnsmasq:x:109:65534:dnsmasq,,,:/var/lib/misc:/bin/false
    sshd:x:110:65534::/var/run/sshd:/usr/sbin/nologin
    pollinate:x:111:1::/var/cache/pollinate:/bin/false

    Reply Report
    • hansen May 10, 2017

      @sergiorivasf40c7ff6ddb06ce

      Well since OS X is based on FreeBSD, which is a Unix-like system like Linux, then there’s no difference - both commands exists on OS X too.
      Your problem is related to Dokku and/or the web server - I think - which you probably didn’t run locally on your Mac.

      Are you using Apache or Nginx as your web server?
      My guess would be you either need to change ownership to www-data or dokku, since I don’t know if it’s related to Dokku or the web server.

      chown -R www-data:www-data /mnt/volume-sgp1-01/

      or

      chown -R dokku:dokku /mnt/volume-sgp1-01/
      Reply Report
      • sergiorivasf40c7ff6ddb06ce May 10, 2017

        thanks so much for helping out like this. Anyway, I tried both www-data and dokku users for the chown command. However, I keep getting the same errors.... In case I’m missing something, here’s the log feed that occurs whenever submitting the avatar.

        2017-05-10T07:52:29.955427961Z app[web.1]: I, [2017-05-10T07:52:29.955169 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"3Wpl78U2ys5SFaTWus8LP+BXNgiQ9x9yJfE9YNw3LKm52sHg5pgpbBApBGZnI28+SZNsS5AOPV0S4ut7I5hstw==", "user"=>{"avatar"=>#<ActionDispatch::Http::UploadedFile:0x00000004fedbd0 @tempfile=#<Tempfile:/tmp/RackMultipart20170510-8-1krj8ya.jpg>, @original_filename="logo.jpg", @content_type="image/jpeg", @headers="Content-Disposition: form-data; name=\"user[avatar]\"; filename=\"logo.jpg\"\r\nContent-Type: image/jpeg\r\n">, "username"=>"student", "email"=>"128828384@qq.com", "password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]", "current_password"=>"[FILTERED]"}, "commit"=>"Update"}
2017-05-10T07:52:29.959778737Z app[web.1]: D, [2017-05-10T07:52:29.959535 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   User Load (0.7ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2  [["id", 1], ["LIMIT", 1]]
2017-05-10T07:52:29.962429632Z app[web.1]: D, [2017-05-10T07:52:29.962156 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   Account Load (0.4ms)  SELECT  "accounts".* FROM "accounts" WHERE "accounts"."subdomain" = $1 LIMIT $2  [["subdomain", "demo"], ["LIMIT", 1]]
2017-05-10T07:52:29.964248470Z app[web.1]: D, [2017-05-10T07:52:29.964019 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   User Load (0.4ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
2017-05-10T07:52:30.126312605Z app[web.1]: D, [2017-05-10T07:52:30.125934 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]    (0.4ms)  BEGIN
2017-05-10T07:52:30.130244793Z app[web.1]: I, [2017-05-10T07:52:30.130053 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: file -b --mime '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cqx438.jpg'
2017-05-10T07:52:30.134487301Z app[web.1]: I, [2017-05-10T07:52:30.134281 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: identify -format '%wx%h,%[exif:orientation]' '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]' 2>/dev/null
2017-05-10T07:52:30.159561414Z app[web.1]: I, [2017-05-10T07:52:30.159239 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: identify -format %m '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]'
2017-05-10T07:52:30.167464542Z app[web.1]: I, [2017-05-10T07:52:30.167236 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: convert '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]' -auto-orient -resize "300x300>" '/tmp/03911f5e177325c649bbb4ce76032ae820170510-8-1ci89q4'
2017-05-10T07:52:30.221948653Z app[web.1]: I, [2017-05-10T07:52:30.221338 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: identify -format '%wx%h,%[exif:orientation]' '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]' 2>/dev/null
2017-05-10T07:52:30.237999724Z app[web.1]: I, [2017-05-10T07:52:30.237646 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: identify -format %m '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]'
2017-05-10T07:52:30.246634422Z app[web.1]: I, [2017-05-10T07:52:30.246396 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: convert '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]' -auto-orient -resize "100x100>" '/tmp/03911f5e177325c649bbb4ce76032ae820170510-8-1pw9n35'
2017-05-10T07:52:30.281802703Z app[web.1]: D, [2017-05-10T07:52:30.281487 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   Account Load (1.0ms)  SELECT  "accounts".* FROM "accounts" WHERE "accounts"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
2017-05-10T07:52:30.284677385Z app[web.1]: D, [2017-05-10T07:52:30.284483 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   Role Load (0.3ms)  SELECT  "roles".* FROM "roles" WHERE "roles"."id" = $1 LIMIT $2  [["id", 5], ["LIMIT", 1]]
2017-05-10T07:52:30.286659115Z app[web.1]: I, [2017-05-10T07:52:30.286442 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: file -b --mime '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-1dh3rii.jpg'
2017-05-10T07:52:30.297516937Z app[web.1]: D, [2017-05-10T07:52:30.297098 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   SQL (2.9ms)  UPDATE "users" SET "avatar_file_name" = $1, "avatar_file_size" = $2, "avatar_updated_at" = $3, "updated_at" = $4 WHERE "users"."id" = $5  [["avatar_file_name", "logo.jpg"], ["avatar_file_size", 18880], ["avatar_updated_at", 2017-05-10 07:52:30 UTC], ["updated_at", 2017-05-10 07:52:30 UTC], ["id", 1]]
2017-05-10T07:52:30.300302400Z app[web.1]: D, [2017-05-10T07:52:30.299993 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]    (0.5ms)  ROLLBACK
2017-05-10T07:52:30.301342636Z app[web.1]: I, [2017-05-10T07:52:30.301102 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Completed 500 Internal Server Error in 346ms (ActiveRecord: 6.6ms)
2017-05-10T07:52:30.303549469Z app[web.1]: F, [2017-05-10T07:52:30.303338 #8] FATAL -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   
2017-05-10T07:52:30.303998659Z app[web.1]: F, [2017-05-10T07:52:30.303473 #8] FATAL -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Errno::EACCES (Permission denied @ dir_s_mkdir - /mnt/volume-sgp1-01):
2017-05-10T07:52:30.304443176Z app[web.1]: F, [2017-05-10T07:52:30.303915 #8] FATAL -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   
2017-05-10T07:52:30.305260336Z app[web.1]: F, [2017-05-10T07:52:30.304730 #8] FATAL -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] vendor/ruby-2.4.0/lib/ruby/2.4.0/fileutils.rb:228:in `mkdir'
        Reply Report
        • hansen May 15, 2017

          @sergiorivasf40c7ff6ddb06ce
          Without looking around in the code, I wouldn’t know why it’s not working.
          I would recommend that you hire a system administrator or developer to setup your mounts correctly and/or modify your code to handle it correctly.
          What’s the ownership and rights of /mnt/volume-sgp1-01? Run this:

          ls -l /mnt/
          Reply Report
      • sergiorivasf40c7ff6ddb06ce May 10, 2017
        [deleted]
        Reply Report
      • sergiorivasf40c7ff6ddb06ce May 11, 2017
        [deleted]
        Reply Report
      • sergiorivasf40c7ff6ddb06ce May 11, 2017

        Unfortunately, the problem still hasn’t been fixed. I am still getting the same error messages. However, as I’m not sure if I missed something or not when reading the error messages, could you take a look at the error logs?

        Link to Error Logs Screenshot

        2017-05-10T07:52:29.955427961Z app[web.1]: I, [2017-05-10T07:52:29.955169 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"3Wpl78U2ys5SFaTWus8LP+BXNgiQ9x9yJfE9YNw3LKm52sHg5pgpbBApBGZnI28+SZNsS5AOPV0S4ut7I5hstw==", "user"=>{"avatar"=>#<ActionDispatch::Http::UploadedFile:0x00000004fedbd0 @tempfile=#<Tempfile:/tmp/RackMultipart20170510-8-1krj8ya.jpg>, @original_filename="logo.jpg", @content_type="image/jpeg", @headers="Content-Disposition: form-data; name=\"user[avatar]\"; filename=\"logo.jpg\"\r\nContent-Type: image/jpeg\r\n">, "username"=>"student", "email"=>"128828384@qq.com", "password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]", "current_password"=>"[FILTERED]"}, "commit"=>"Update"}
2017-05-10T07:52:29.959778737Z app[web.1]: D, [2017-05-10T07:52:29.959535 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   User Load (0.7ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2  [["id", 1], ["LIMIT", 1]]
2017-05-10T07:52:29.962429632Z app[web.1]: D, [2017-05-10T07:52:29.962156 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   Account Load (0.4ms)  SELECT  "accounts".* FROM "accounts" WHERE "accounts"."subdomain" = $1 LIMIT $2  [["subdomain", "demo"], ["LIMIT", 1]]
2017-05-10T07:52:29.964248470Z app[web.1]: D, [2017-05-10T07:52:29.964019 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   User Load (0.4ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
2017-05-10T07:52:30.126312605Z app[web.1]: D, [2017-05-10T07:52:30.125934 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]    (0.4ms)  BEGIN
2017-05-10T07:52:30.130244793Z app[web.1]: I, [2017-05-10T07:52:30.130053 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: file -b --mime '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cqx438.jpg'
2017-05-10T07:52:30.134487301Z app[web.1]: I, [2017-05-10T07:52:30.134281 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: identify -format '%wx%h,%[exif:orientation]' '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]' 2>/dev/null
2017-05-10T07:52:30.159561414Z app[web.1]: I, [2017-05-10T07:52:30.159239 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: identify -format %m '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]'
2017-05-10T07:52:30.167464542Z app[web.1]: I, [2017-05-10T07:52:30.167236 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: convert '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]' -auto-orient -resize "300x300>" '/tmp/03911f5e177325c649bbb4ce76032ae820170510-8-1ci89q4'
2017-05-10T07:52:30.221948653Z app[web.1]: I, [2017-05-10T07:52:30.221338 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: identify -format '%wx%h,%[exif:orientation]' '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]' 2>/dev/null
2017-05-10T07:52:30.237999724Z app[web.1]: I, [2017-05-10T07:52:30.237646 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: identify -format %m '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]'
2017-05-10T07:52:30.246634422Z app[web.1]: I, [2017-05-10T07:52:30.246396 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: convert '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-cyc8ia.jpg[0]' -auto-orient -resize "100x100>" '/tmp/03911f5e177325c649bbb4ce76032ae820170510-8-1pw9n35'
2017-05-10T07:52:30.281802703Z app[web.1]: D, [2017-05-10T07:52:30.281487 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   Account Load (1.0ms)  SELECT  "accounts".* FROM "accounts" WHERE "accounts"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
2017-05-10T07:52:30.284677385Z app[web.1]: D, [2017-05-10T07:52:30.284483 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   Role Load (0.3ms)  SELECT  "roles".* FROM "roles" WHERE "roles"."id" = $1 LIMIT $2  [["id", 5], ["LIMIT", 1]]
2017-05-10T07:52:30.286659115Z app[web.1]: I, [2017-05-10T07:52:30.286442 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Command :: file -b --mime '/tmp/96d6f2e7e1f705ab5e59c84a6dc009b220170510-8-1dh3rii.jpg'
2017-05-10T07:52:30.297516937Z app[web.1]: D, [2017-05-10T07:52:30.297098 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   SQL (2.9ms)  UPDATE "users" SET "avatar_file_name" = $1, "avatar_file_size" = $2, "avatar_updated_at" = $3, "updated_at" = $4 WHERE "users"."id" = $5  [["avatar_file_name", "logo.jpg"], ["avatar_file_size", 18880], ["avatar_updated_at", 2017-05-10 07:52:30 UTC], ["updated_at", 2017-05-10 07:52:30 UTC], ["id", 1]]
2017-05-10T07:52:30.300302400Z app[web.1]: D, [2017-05-10T07:52:30.299993 #8] DEBUG -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]    (0.5ms)  ROLLBACK
2017-05-10T07:52:30.301342636Z app[web.1]: I, [2017-05-10T07:52:30.301102 #8]  INFO -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Completed 500 Internal Server Error in 346ms (ActiveRecord: 6.6ms)
2017-05-10T07:52:30.303549469Z app[web.1]: F, [2017-05-10T07:52:30.303338 #8] FATAL -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   
2017-05-10T07:52:30.303998659Z app[web.1]: F, [2017-05-10T07:52:30.303473 #8] FATAL -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] Errno::EACCES (Permission denied @ dir_s_mkdir - /mnt/volume-sgp1-01):
2017-05-10T07:52:30.304443176Z app[web.1]: F, [2017-05-10T07:52:30.303915 #8] FATAL -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991]   
2017-05-10T07:52:30.305260336Z app[web.1]: F, [2017-05-10T07:52:30.304730 #8] FATAL -- : [0811165f-2b5b-4a82-8b67-2c6da68ce991] vendor/ruby-2.4.0/lib/ruby/2.4.0/fileutils.rb:228:in `mkdir'

        Thank you so much for all your help!

        edited by kamaln7
        Reply Report
        • hansen May 11, 2017

          @sergiorivasf40c7ff6ddb06ce

          I don’t see a lot of information besides a “Permission denied” when the code is trying to create a new directory (mkdir).

          So if you change your code to upload somewhere else - what’s the ownership and rights of files you upload there?

          What is the user and group of your app/web-server or whatever is handling the upload?

          What is the current user/group and rights of /mnt/volume-sgp1-01?

          Reply Report
hansen May 12, 2017

Hi @sergiorivasf40c7ff6ddb06ce
I would recommend that you hire a system administrator to setup and maintain your server.
I know @jtittle is an excellent administrator and you can contact him if you want:
https://www.digitalocean.com/community/users/jtittle

Reply Report
Submit an Answer

Related

Looking for something else?

Ask a new question Search for more help