AWS recommends setting all keepalive timeout configurations to be greater than the idle timeout. This is longer than our configured ELB idle timeout of 60 seconds. Get the latest and greatest from MDN delivered straight to your inbox. A quick look over our Nginx configurations showed that the keepalive connections were set to 75s. Learn the best of web development. This post will describe how we were able to get to the bottom of these mysterious 504 errors and what others can do to prevent them in the future.These timeout errors were hard to diagnose since the associated requests did not show up as errors in our API service logs. The website is currently inaccessible to users. For example, using Apache as backend server and Nginx as front-end proxy server. If you attempt to visit the website you get a 504 Gateway Time-out.

Since our ELB idle timeout is less than our keepalive timeout, the connections between the Nginx proxies and the ELB should remain open for at least the ELB’s idle timeout. This means that Explicitly configuring Nginx with a client_header_timeout value than our ELB’s idle timeout was sufficient in eliminating the 504 errors. Since our body transfer doesn’t typically take that long, this configuration shouldn’t affect TCP connectivity.After increasing client_header_timeout to 75s, Nginx was no longer sending TCP RST packets at 10s intervals. The newsletter is offered in English only at the moment. I recorded the TCP activity on the machine and saved it to a packet capture file for inspection:After copying the packet capture to my machine, I inspected its contents with . In order to provide uptime that meets the demanding needs of our customers, we use AWS’ “Classic” Elastic Load Balancer (ELB) to distribute incoming requests among several AWS instances. These AWS instances host Nginx reverse proxies which pass requests to our SigOpt API services. Very frustrating! Apart from a web server, Nginx can also be configured as a proxy server. For example, when working with php-fpm and other dynamic language modules. From what I had observed so far, the second situation seemed more likely:There was no surge in CPU/memory load or request length on the instances when the 504 errors occurred.The ELB would respond with a 504 almost immediately, much faster than the ELB’s idle timeout and the processing time for an average request.The timeouts were much more frequent than service or machine restarts.Failed requests often couldn’t be traced from the load balancers to the corresponding backend service.The second case can occur when a server running on a registered instance is incorrectly configured. Requests will sometimes fail for reasons out of our control, but we wanted to make sure we were doing everything on our end to … You can change this value at your discretion. Same here, have had a number of flows running perfectly well until the end of last week. Nginx defines the client_header_timeout as being measured . Angular PWA app getting 504 (Gateway Timeout) calling Node JS/Express API, but only when running from device.

Why sign up for an OpenEye account? Literally, it means that the response time from the server has been exceeded. Our research team is constantly developing new optimization techniques for real-world problems. Infrastructure diagram for SigOpt API staging environment. With this configuration, the instance should close the connection after 10 seconds if client_header_timeout is set to 10s, or 75 seconds if it is set to 75s.Using the original Nginx configurations (client_header_timeout = 10s), the connection remained established for only 10 seconds before being closed by the instance:After updating the client_header_timeout to 75s, TCP connections were kept open for 75s rather than 10s. Meanwhile, there may be following reasons of 504 gateway timeout: Browser errors – occurs when browser has some errors in configurations or settings. During this investigation, the number of registered instances was 2 (n=2).In April, we noticed that a small number of API requests were timing out, returning 504 (Gateway Timeout) errors to users. The ELB will remain unaware of this closed connection and proceed as if the connection is still open. Since this timeout is only set , this would require that the ELB take greater than 10 seconds to transmit request data to the registered instance. In the log, you can see not only the scripts themselves but could also trace the root cause that led to such problems.Then you just have to figure out what to do with it, for example, optimize scripts or disable extra plug-ins.In this article, we examined how to fix 504 gateway timeout issue faced in Nginx, and also why this error may occur.Useful articles, tips and videos about creating and promoting websites to your mail Learn more about optimization and how SigOpt augments practitioners in the modeling workflow.