I uninstalled it in the following manner, use find to see where gitlab is hiding:
$ find / -iname "gitlab"

Once you find the right folder, try to execute the gitlab-ctl binary as follows:

$ root@centos [/home/ryan]# /opt/gitlab/bin/gitlab-ctl uninstall
ok: down: nginx: 29s, normally up
ok: down: postgresql: 28s, normally up
ok: down: redis: 28s, normally up
ok: down: sidekiq: 27s, normally up
ok: down: unicorn: 11s, normally up
could not find nginx runit pidfile (service already stopped?), cannot attempt SIGKILL...
ok: down: nginx: 29s, normally up
could not find postgresql runit pidfile (service already stopped?), cannot attempt SIGKILL...
ok: down: postgresql: 28s, normally up
could not find redis runit pidfile (service already stopped?), cannot attempt SIGKILL...
ok: down: redis: 28s, normally up
could not find sidekiq runit pidfile (service already stopped?), cannot attempt SIGKILL...
ok: down: sidekiq: 27s, normally up
could not find unicorn runit pidfile (service already stopped?), cannot attempt SIGKILL...
ok: down: unicorn: 11s, normally up
Your config files have been backed up to /root/gitlab-cleanse-2014-04-29T19:05.


The binary location may be different, if you have trouble, post your output to the find script. Good luck!

How did you install GitLab? Did you follow a guide on doing that?

Note, I had to reboot to kill the processes that were currently on. I tried killing them with -9, however this was not a production server so it was worth rebooting.