Related

Product App Platform: Free static site hosting Product
PHP Error Log: authz_core:error - For files that don't exists on my server Question Question
blocked ip in spam Question Question

Question

Javascript error in precompiled asset: "Invalid or unexpected token" for code '#!/usr/bin/env node'

Posted May 23, 2017 9k views
Ruby on RailsLinux BasicsUbuntu 16.04JavaScript

My site runs fine locally in development. However, deployed on digital ocean in the Ubuntu 16.04 environment, it breaks because the so-called ‘shebang’ line above, which is not in any of my original js code, nor directly in any of the js libraries I have incorporated into my project, errors in my browser console. The “#!/usr/bin/env node’ line is line #29589 in a precompiled asset.

My assets were precompiled locally an are included in my repo. I removed them all from my Production host and ran rake assets:precompile there, but still get the error. I manually removed the line from the precompiled file in question, restarted unicorn and nginx, yet still get the error, which I completely don’t understand.

This isn’t my code, but it’s somehow breaking my production site for unknown reasons, and I doubt it should be.

Can anyone explain to me:

  1. Is the line ’#!/usr/bin/env node’ supposed to be in my precompiled asset or not.
  2. If it is, why it breaks my production site, or, if it ain’t, why it’s there and how to make it not appear in my precompiled asset.
  3. what am I doing wrong in troubleshooting that manually deleting the offending line, then clearing my local cache and manually restarting unicorn and nginx on my droplet STILL results in the removed line appearing and breaking my site in my local broswer?
Add a comment
jcspampinato
By:
jcspampinato

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
3 answers
jtittle1 May 23, 2017

@jcspampinato

The shebang is required if the NodeJS script is required to be executed from the CLI, without using node. It functions much in the same way the shebang does for a bash script.

For example, let’s create two very basic NodeJS scripts:

demo01.js – w/o a shebang

console.log("Hello World");

demo02.js – w/ a shebang

#!/usr/bin/env node
console.log("Hello World");

If we run either demo01.js or demo02.js from the CLI using either:

node demo01.js

or

node demo02.js

… we should see “Hello World” as the output. So why use a shebang in demo02.js? Well, if it needs to be ran from the CLI, like a bash script, it’s required – additionally, execution permissions would be set to allow it to be executed in that manor.

For example, run chmod +x demo01.js and then chmod +x demo02.js. That’ll give the scripts their required permissions to execute as a script from the CLI, without node.

Now run (without using node):

./demo01.js

and

./demo02.js

demo01.js fails with:

./demo01.js: line 1: syntax error near unexpected token `"Hello World"'
./demo01.js: line 1: `console.log("Hello World");'

… while demo02.js results in the same “Hello World” that was expected. The reason for this is because that shebang equates to running node demo02.js.

…

So the question becomes does one or more of the scripts that you’re compiling require the ability to execute as essentially a shell script (like demo02.js), and if so, is pre-compiling preventing that?

Also, are you 100% sure that the environments are identical? Some use node, others use nodejs. If you have either or and the shebang references the one you don’t have an executable for, it’ll fail.

Reply Report
  • jtittle1 May 23, 2017

    @jcspampinato

    You should be able to run:

    which node

    or

    which nodejs

    .. in both environments to see what executables exist. For example, using NodeSource’s repos:

    which node => /usr/bin/node

    which nodejs => /usr/bin/nodejs

    However, when running:

    ls -al /usr/bin/node

    I see:

    lrwxrwxrwx 1 root root 22 May 23 02:32 /usr/bin/node -> /etc/alternatives/node

    Which means /usr/bin/node is symlink'ed to /etc/alternatives/node and it’s symlink'ed to /usr/bin/nodejs, so they have backwards compatibility in place.

    /usr/bin/node => /etc/alternatives/node => /usr/bin/nodejs

    That means I can run node demo01.js or nodejs demo.js – not all repositories set that up, or allow for it unless you do your own custom symlink'ing.

    Reply Report
    • jcspampinato May 23, 2017

      Thanks for all of the info.

      In both environments, ‘which node’ results in ’/usr/local/bin/node’.

      However,

      I’ve verified that the VPS production environment’s copy in /usr/local/bin/node symlinks all the way back to /usr/bin/nodejs,

      whereas my development environment only has it directly installed in /usr/local/bin/node.

      My develoment environment has no copy of nodejs, whereas 'which nodejs’ returns ’/usr/bin/nodejs’.

      So a little research
      here and here seem to indicate that node and nodejs are completely different things.

      But 'node -h’ in my development environment confirms that the local node is 'node.js’, and -v reports v4.4.7.

      Production version is v6.10.3.

      I’m skeptical that the version mismatch is the source of my problem.

      And I don’t really know how I can answer this:

      So the question becomes does one or more of the scripts that you’re compiling require the ability to execute as essentially a shell script (like demo02.js), and if so, is pre-compiling preventing that?

      Clearly the shebang indicates that the script is expected to run as a shell script. I don’t know enough to know how or why precompiling would break that.

      Reply Report
      • jcspampinato May 23, 2017

        Downgrading the production server to node v4.4.7 has no effect.

        Nothing I do seems to affect the error. I have physically removed the line from the offending precompiled file and deleted the accompanying same-named.gz from my production repo, then rebooted the VPS and reloaded the site in my browser, and I still get the javascript error on the deleted line - that is physically not present in my VPS repo - in my console. I deleted my local cache as well.

        Where would this be cached on my server or locally? Why am I getting the error after removing the shebang from the precompiled asset? How else can I troubleshoot this problem?

        Reply Report
        • jtittle1 May 23, 2017

          @jcspampinato

          Did you also post this on StackOverflow by chance?

          If so, and this is the snippet:

          #!/usr/bin/env node
require('coffee-script/register')
require('./node_modules/vows/bin/vows')

;
var concatMap = require('../');
var xs = [ 1, 2, 3, 4, 5, 6 ];
var ys = concatMap(xs, function (x) {
    return x % 2 ? [ x - 0.1, x, x + 0.1 ] : [];
});
console.dir(ys);

          The only real issue I can see there is the stray ;, which could be throwing things off.

          i.e.

          require('coffee-script/register')
require('./node_modules/vows/bin/vows')

;

          If you remove the stray ; and change the require() lines to:

          require('coffee-script/register');
require('./node_modules/vows/bin/vows');

          Does that change anything?

          Beyond that, I’m a bit stumped as Node should ignore that line (as shown in my examples). The only time it’d actually be parsed is if you were actually giving the compiled application execution permissions and running without the node prefix.

          Reply Report
          • jcspampinato May 23, 2017

            Yes, that’s me on stackoverflow, and that’s the snippet. I’ll try your recommended changes to the compiled asset.

            However, it turns out that the assets may have indeed precompiled incorrectly; If I run ‘RAILS_ENV=production rake assets:precompile’ I get the failure trace listed at the bottom of this reply. I can’t tell for certain what file and line cause the problem from this info, but that the error is a ’#’ character is a good sign that it’s the same offending line in the compiled asset.

            I’m embarrassed to say that after hours of attempting to get my site running, I blindly attempted to implement a solution implied by this what appeared to be similar problem. My asset precompiles without error if I run 'RAILSENV=production;FEECALC=2 rake assets:precompile’, but that produces the asset that errors out when the site loads. FEE_CALC is an environment variable specific to the code in the example and has nothing to do with my project; I did not take the time to look too hard at why it worked when it did, as it was the only thing I had tried up until then that worked at all.

            So the problem definition now seems to be that my assets fail to precompile with the following error, how can I confirm whether this is the problematic pulled in shebang, or otherwise recitfy the issue? I plan to confirm I have no stray ’#’ in my javascript elsewhere.

            The trace is:

            (in /home/rails/popEFX)
            rake aborted!
            ExecJS::RuntimeError: SyntaxError: Unexpected character ’#’
            JSParseError.get ((execjs):3538:621)
            (execjs):4060:47
            (execjs):1:102
            Object.<anonymous> ((execjs):1:120)
            Module.compile (module.js:570:32)
            Object.Module.            extensions..js (module.js:579:10)
            Module.load (module.js:487:32)
            tryModuleLoad (module.js:446:12)
            Function.Module.load (module.js:438:3)
            Module.runMain (module.js:604:10)
            /usr/local/rvm/gems/ruby-2.4.0/gems/execjs-2.7.0/lib/execjs/external            runtime.rb:39:in exec'
            /usr/local/rvm/gems/ruby-2.4.0/gems/execjs-2.7.0/lib/execjs/external_runtime.rb:21:in            eval’
            /usr/local/rvm/gems/ruby-2.4.0/gems/execjs-2.7.0/lib/execjs/externalruntime.rb:46:in call'
            /usr/local/rvm/gems/ruby-2.4.0/gems/uglifier-3.2.0/lib/uglifier.rb:195:in            run            uglifyjs’
            /usr/local/rvm/gems/ruby-2.4.0/gems/uglifier-3.2.0/lib/uglifier.rb:157:in compile'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/uglifier_compressor.rb:53:in            call’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/uglifiercompressor.rb:28:in `call’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/processor            utils.rb:75:in call_processor'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/processor_utils.rb:57:in            block in callprocessors’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/processor            utils.rb:56:in reverse_each'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/processor_utils.rb:56:in            callprocessors’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/loader.rb:134:in `load            fromunloaded’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/loader.rb:60:in block in load'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/loader.rb:317:in            fetch            assetfromdependencycache’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/loader.rb:44:in `load’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/cached            environment.rb:20:in block in initialize'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/cached_environment.rb:47:in            load’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/base.rb:66:in find_asset'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/base.rb:73:in            findalllinkedassets’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/manifest.rb:142:in block in find'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/legacy.rb:114:in            block (2 levels) in logical            paths’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/pathutils.rb:228:in `block in stattree’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/pathutils.rb:212:in `block in statdirectory’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/pathutils.rb:209:in `each’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/path            utils.rb:209:in stat_directory'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/path_utils.rb:227:in            stattree’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/legacy.rb:105:in each'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/legacy.rb:105:in            block in logical            paths’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/legacy.rb:104:in each'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/legacy.rb:104:in            logicalpaths’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/manifest.rb:140:in find'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/sprockets/manifest.rb:185:in            compile’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-rails-3.2.0/lib/sprockets/rails/task.rb:68:in block (3 levels) in define'
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-3.7.1/lib/rake/sprocketstask.rb:147:in            with            logger’
            /usr/local/rvm/gems/ruby-2.4.0/gems/sprockets-rails-3.2.0/lib/sprockets/rails/task.rb:67:in block (2 levels) in define'
            /usr/local/rvm/gems/ruby-2.4.0@global/gems/rake-12.0.0/exe/rake:27:in            <top (required)>’
            /home/rails/railsproject/vendor/bundle/bin/rubyexecutablehooks:15:in `eval’
            /home/rails/rails            project/vendor/bundle/bin/rubyexecutablehooks:15:in `<main>’
            Tasks: TOP => assets:precompile
            (See full trace by running task with –trace)

            Reply Report
jtittle1 May 23, 2017

@jcspampinato

In most all scenarios, !#/usr/bin/env node should work just fine as it should find the executable. I say should – but that’s definitely not the case here.

It’s true that on older distros, node and nodejs were two entirely different packages. As of Ubuntu 16.04/16.10, the node package that was causing the confusion seems to have been pulled.

Differences in version can definitely cause some issues, I can confirm that from answering questions here in the community. What I might recommend doing would be to setup a test Droplet and use the repositories from NodeSource.

You can installer 4.x using:

curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash -

apt-get install -y nodejs

You could then test the precompiled application against a 4.x release and see if it works. If it does, it may very well be a matter of versions (something was deprecated, removed, etc). If it doesn’t work, the issue may be between environments.

That’s what I’d do at least to test. I’d also take a look at the packages that are being installed during the compile. I’ve ran in to numerous abandoned packages and packages that hadn’t been updated for over a year with one question – they had no signs of coming back to update either.

Reply Report
jtittle1 May 23, 2017

@jcspampinato

New reply due to response limits on the previous.

…

I would try to find the package that’s being compiled in which has the shebang, and then try to run without it to test whether it’s related to the problem, and thus, the error.

I mainly work with PHP/MySQL and Bash/Shell scripting, though similar issues exist when compiling NGINX and then using include to keep configuration files clean. You’ll often see an error that exists somewhere on the fully included source, but since you’re using includes, you have to manually open each file to find out where the issue may be (as the errors don’t correspond to included files).

That seems to be a similar case here. When you build all the assets together, it throws and error and fails, much like how NGINX does with includes at times, thus you have to do a lot of manual work to find out where the source of the issue really is.

So I’d try to find what has the shebang and compile without the package first. If it succeeds, you’ve found the issue. If it doesn’t there may be other issues that exist.

Reply Report
Submit an Answer

Related

Looking for something else?

Ask a new question Search for more help