502badgatewayGitlaberrorinnginxserver

BeforedescribingthediagnosisandtroubleshootingmethodsthatIusedtosolvethisproblem,allowmeto
giveyoualittlerundownonwhatthisisallabout.

GitHub: It is a wonderful tool that make managing and administering lots of Git repositories and their
associated permissions on a centralized server quite easy.
What if you want to do something like this on your own servers where your developers can store their
projects and network?
GitLab gives you complete control over your repositories and allows you to decide whether they are
public, private or free. Its a github clone to be more specific running on your own hardware.
To learn more about the minimum requirements to run your own gitlab service click here.
Hunting down the errors with GitLab:
One of our gitlab servers started throwing up 502 bad gateway error one fine day. We have a
Ubuntu server with nginx+gitlab+ISPconfig (ISPconfig is a free hosting control panel) to server as a
gitlab server.
Troubleshooting:
The usual routine is to check the error logs pertaining to the service.
We found this from gitlab_erorr.log
2012/09/21
23:42:29
[error]
19689#0:
*5
connect()
to
unix:/home/gitlab/gitlab/tmp/sockets/gitlab.socket failed (111: Connection refused) while
connecting to upstream, client: ****, server: gitlab..***, request: GET / HTTP/1.1,
upstream:
Bad Gateway means nginx wasnt able to connect to the upstream socket.
Three possible reasons:1. Misconfigured path to the socket
2. Unicorn is not running.
3. Unicorn crashes intermittently due to resource shortage
There was no resource shortage. Server was in ship-shape.
We tried to restart gitlab
root@server:/etc/nginx/sites-available# /etc/init.d/gitlab restart
Removing stale Unicorn web server pid. This is most likely caused by the web server
crashing
the
last
time
it
ran.
Removing stale Sidekiq web server pid. This is most likely caused by the Sidekiq crashing
the
last
time
it
ran.
Starting
the
GitLab
Unicorn
web
server
master
failed
to
start,
check
stderr
log
for
details
Starting
the
GitLab
Sidekiq
event
dispatcher
The
GitLab
Unicorn
webserver
is
not
running.
The GitLab Sidekiq job dispatcher with pid 6296 is running.

The solution still eluded us. After a couple of minutes googling using The GitLab Unicorn webserver
is not running as search keyword. We stumbled on the possibility of a port number
mismatch/misconfiguration. We decided to check that.
Root Cause
We noticed that ISPconfig and Unicorn were running on same port 8080. Now that is the root cause of
this problem. One of our associates had installed ISPconfig couple of hours before and inadvertently
used the port number 8080 reserved for unicorn.
We changed ISP config port to 8081 from here:/etc/nginx/sites-available/ispconfig.vhost
/home/git/gitlab/config/unicorn.rb
Restarted ngnix and gitlab
Now both services are no longer sharing one port.
root@server:/home/git/gitlab/config# netstat -tulpn | grep :808
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 6692/unicorn.rb -E
tcp 0 0 0.0.0.0:8081 0.0.0.0:* LISTEN 6627/nginx
So, that was it! Double check before and after you install anything on the server. Make sure it does not
adversely affect or tamper the functions of other services.