Nginx Admin's Handbook

My notes on NGINX administration basics, tips & tricks, caveats, and gotchas.

^{Hi-diddle-diddle, he played on his

fiddle and danced with lady pigs.

Number three said, "Nicks on tricks!

I'll build my house with EN-jin-EKS!".

The Three Little Pigs: Who's Afraid of the Big Bad Wolf?}

HTTP Basics
SSL/TLS Basics
NGINX Basics
Helpers
Base Rules (16)
Debugging (5)
Performance (13)
Hardening (31)
Reverse Proxy (8)
Load Balancing (2)
- Tweak passive health checks
- Don't disable backends by comments, use down parameter
Others (4)
Configuration Examples
- Reverse Proxy

Introduction

Before you start playing with NGINX please read an official Beginner’s Guide. It's a great introduction for everyone.

Nginx (/ˌɛndʒɪnˈɛks/ EN-jin-EKS, stylized as NGINX or nginx) is an open source HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server with a strong focus on high concurrency, performance and low memory usage. It is originally written by Igor Sysoev.

For a long time, it has been running on many heavily loaded Russian sites including Yandex, Mail.Ru, VK, and Rambler. At this moment some high-profile companies using NGINX include Cisco, DuckDuckGo, Facebook, GitLab, Google, Twitter, Apple, Intel, and many more. In the September 2019 it was the most commonly used HTTP server (see Netcraft survey).

NGINX is a fast, light-weight and powerful web server that can also be used as a:

fast HTTP reverse proxy
reliable load balancer
high performance caching server
full-fledged web platform

So, to be brief, it provides the core of complete web stacks and is designed to help build scalable web applications. When it comes to performance, NGINX can easily handle a huge amount of traffic. The other main advantage of the NGINX is that allows you to do the same thing in different ways.

Unlike traditional HTTP servers, NGINX doesn't rely on threads to handle requests and it was written with a different architecture in mind - one which is much more suitable for nonlinear scalability in both the number of simultaneous connections and requests per second.

NGINX is also known as a Apache Killer (mainly because of its lightness and much less RAM consumption). It is event-based, so it does not follow Apache's style of spawning new processes or threads for each web page request. Generally, it was created to solve the C10K problem.

For me, it is a one of the best and most important service that I used in my SysAdmin career.

These essential documents should be the main source of knowledge for you:

Getting Started
NGINX Documentation
Development guide
Security Controls

In addition, I would like to recommend three great docs focuses on the concept of the HTTP protocol:

HTTP Made Really Easy
Hypertext Transfer Protocol Specification
Web technology for developers - HTTP

If you love security keep your eye on this one: Cryptology ePrint Archive. It provides access to recent research in cryptology and explores many subjects of security (e.g. Ciphers, Algorithms, SSL/TLS protocols). A great introduction that covers core concepts of cryptography is Practical Cryptography for Developers. I also recommend to read the Bulletproof SSL and TLS. Yep, it's definitely the most comprehensive book about deploying TLS for me.

An obligatory source of knowledge is also the OWASP Cheat Sheet Series. You should ought treat it as an excellent security guidance. Burp Scanner - Issue Definitions introduces you to the web apps and security vulnerabilities. Finally, The Web Security Academy is a free online training center for web application security with high-quality reading materials and interactive labs of varying levels of difficulty. All are really good source to start learning about web application security.

And, of course, always browse official Nginx Security Advisories and CVE databases like CVE Details or CVE - The MITRE Corporation - to stay Up-to-Date on NGINX vulnerabilities.

Prologue

When I was studying architecture of HTTP servers I became interested in NGINX. As I was going through research, I kept notes. I found a lot of information about it, e.g. forum posts on the web about every conceivable problem was great. However, I've never found one guide that covers the most important things in a suitable form. I was a little disappointed.

I was interested in everything: NGINX internals, functions, security best practices, performance optimisations, tips & tricks, hacks and rules, but for me some of the documents treated the subject lightly.

Of course, NGINX Official Documentation is the best place but I know that we also have other great resources:

agentzh's Nginx Tutorials
Nginx Guts
Nginx discovery journey
Nginx Secure Web Server
Emiller’s Guide To Nginx Module Development
Emiller’s Advanced Topics In Nginx Module Development

These are definitely the best assets for us and in the first place you should seek help there. Moreover, in order to improve your knowledge, please see Books chapter - it contains top literature on NGINX.

Why I created this handbook

For me, however, there hasn't been a truly in-depth and reasonably simple cheatsheet which describe a variety of configurations and important cross-cutting topics for HTTP servers. Configuration of the NGINX can be tricky sometimes and you really need to get into the syntax and concepts to get an understanding tricks, loopholes, and mechanisms. The documentation isn't as pretty as other projects and should certainly include more robust examples.

This handbook is a set of rules and recommendations for the NGINX Open Source HTTP server. It also contains the best practices, notes, and helpers with countless examples. Many of them refer to external resources.

There are a lot of things you can do to improve in your NGINX instance and this guide will attempt to cover as many of them as possible. For the most part, it contains the most important things about NGINX for me. I think the configuration you provided should work without any talisman. That's why I created this repository.

With this handbook you will explore the many features and capabilities of the NGINX. You'll find out, for example, how to testing the performance or how to resolve debugging problems. You will learn configuration guidelines, security design patterns, ways to handle common issues and how to stay out of them. I explained here a few best tips to avoid pitfalls and configuration mistakes.

I added set of guidelines and examples has also been produced to help you administer of the NGINX. They give us insight into NGINX internals also.

Mostly, I apply the rules presented here on the NGINX working as a reverse proxy. However, does not to prevent them being implemented for NGINX as a standalone server.

Who this handbook is for

If you do not have the time to read hundreds of articles (just like me) this multipurpose handbook may be useful. I created it in the hope that it will be useful especially for System Administrators and Experts of Web-based applications.

This handbook does not get into all aspects of NGINX. What's more, some of the things described in this guide may be rather basic because most of us do not configure NGINX every day and it is easy to forget about basic/trivial things. On the other hand, also discusses heavyweight topics so there is something for advanced users. I tried to put external resources in many places in this handbook in order to dispel any suspicion that may exist.

I did my best to make this handbook a single and consistent (but now I know that is really hard). It's organized in an order that makes logical sense to me. I think it can also be a good complement to official documentation and other great documents. Many of the topics described here can certainly be done better or different. Of course, I still have a lot to improve and to do. I hope you enjoy and have fun with it.

Do not treat this handbook and notes written here as revealed knowledge. You should take a scientific approach when reading this document. If you have any doubts and disagree with me, please point out my mistakes. You should to discover cause and effect relationships by asking questions, carefully gathering and examining the evidence, and seeing if all the available information can be combined in to a logical answer.

I create this handbook for one more reason. Rather than starting from scratch in, I putting together a plan for answering your questions to help you find the best way to do things and ensure that you don't repeat my mistakes from the past.

So, what's most important:

ask a questions about something that you observe
do background research
do tests with an experiments
analyze and draw conclusions
communicate results (for us!)

Finally, you should know I'm not a NGINX expert but I love to know how stuff works and why work the way they do. I’m not a crypto expert... but I do know the term "elliptic curve" (I really like this quote!). Don't need to be an expert to figure out the reason just got to have used this and not this or why something works this way and not another. It feels good to understand the recommendations and nuances of a topic you’re passionate about.

Before you start

Remember about the following most important things:

Blindly deploying of the rules described here can damage your web application!

Do not follow guides just to get 100% of something. Think about what you actually do at your server!

Copy-and-paste is not the best way to learn. Think twice before adopting rules from this handbook.

There are no settings that are perfect for everyone.

Always think about what is better and more important for you: security vs usability/compatibility.

Security mainly refers to minimise the risk.

Change one thing may open a whole new set of problems.

Read about how things work and what values are considered secure enough (and for what purposes).

The only correct approach is to understand your exposure, measure and tune.

+ Security is important for ethical reasons. Compliance is important for legal reasons.
+ The key to workplace contentment is understanding they are unrelated to each other.
+ Both are important, but one does not lead to the other (compliance != security).
author: unknown

+ Security is always needed, no matter what type of website it is. It can be static HTML
+ or fully dynamic, an attacker can still inject hostile content into the page in transit
+ to attack the user.
author: Scott Helme

+ Don’t enable older deprecated protocols just because Karen in Florida is still using
+ a PC that she bought back in 2001.
author: thisinterestsmeblog

I think, in the age of phishing, cyber attacks, ransomware, etc., you should take care of security of your infrastructure as hard as possible but don't ever forget about this one...

Lastly, I would like to quote two very important comments found on the web about compliance with the standards and regulations, and essence of a human factor in security:

Regulations that make sense are often not descriptive - capturing the intent and scope of a rule often requires technical expertise. More than that, it's the type of expertise most organisations do not have. And instead of improving themselves, these companies, who may form the grand majority of the industry, petition the regulators to provide a safe checklist of technical mitigations that can be implemented to remain compliant. [...] Instead of doing the right thing and meeting the planned intent, companies are instead ticking nonsensical boxes that the regulators and their auditors demand. Blindly. Mindlessly. Divorced from reality. - by bostik

Whenever considering security, the human factor is nearly always as important or more important than just the technical aspects. Policy and procedures need to consider the human element and try to ensure that these policies and procedures are structured in such a way as to help enable staff to do the right thing, even when they may not fully understand why they need to do it. - by Tim X

Contributing & Support

A real community, however, exists only when its members interact in a meaningful way that deepens their understanding of each other and leads to learning.

If you find something which doesn't make sense, or something doesn't seem right, please make a pull request and please add valid and well-reasoned explanations about your changes or comments.

Before adding a pull request, please see the contributing guidelines.

Code Contributors

This project exists thanks to all the people who contribute.

ToDo

What needs to be done? Look at the following ToDo list:

New chapters:

Bonus Stuff
HTTP Basics
SSL/TLS Basics
Reverse Proxy
Caching
Core modules
3rd party modules
Web Application Firewall
ModSecurity
Debugging

Existing chapters:

Introduction

Prologue
Why I created this handbook
Who this handbook is for
Before you start
Contributing & Support
_RSS Feed & Updates
Checklist to rule them all

Bonus Stuff

Fully automatic installation
Static error pages generator
Server names parser

Books

ModSecurity 3.0 and NGINX: Quick Start Guide
Cisco ACE to NGINX: Migration Guide

External Resources

Nginx official
- Nginx Forum
- Nginx Mailing List
- NGINX-Demos
Presentations & Videos
- NGINX: Basics and Best Practices
- NGINX Installation and Tuning
- Nginx Internals (by Joshua Zhu)
- Nginx internals (by Liqiang Xu)
- How to secure your web applications with NGINX
- Tuning TCP and NGINX on EC2
- Extending functionality in nginx, with modules!
- Nginx - Tips and Tricks.
- Nginx Scripting - Extending Nginx Functionalities with Lua
- How to handle over 1,200,000 HTTPS Reqs/Min
- Using ngx_lua / lua-nginx-module in pixiv
Cheatsheets & References
- Nginx configurations for most popular CMS/CMF/Frameworks based on PHP
Performance & Hardening
- Memorable site for testing clients against bad SSL configs
Config parsers
- Quick and reliable way to convert NGINX configurations into JSON and back
- Parses nginx configuration with Pyparsing
Config managers
- Ansible role to install and manage nginx configuration
- Ansible Role - Nginx
- Ansible role for NGINX
- Puppet Module to manage NGINX on various UNIXes
Static analyzers
- nginx-minify-conf
Comparison reviews
- NGINX vs. Apache (Pro/Con Review, Uses, & Hosting for Each)
- Web cache server performance benchmark: nuster vs nginx vs varnish vs squid
Builder tools
- Nginx-builder
Benchmarking tools
- wrk2
- httperf
- slowloris
- slowhttptest
- GoldenEye
Debugging tools
- strace
- GDB
- SystemTap
- stapxx
- htrace.sh
Security & Web testing tools
- Burp Suite
- w3af
- nikto
- ssllabs-scan
- http-observatory
- testssl.sh
- sslyze
- cipherscan
- O-Saft
- Nghttp2
- h2spec
- http2fuzz
- Arjun
- Corsy
- XSStrike
Online & Web tools
- ssltools
Other stuff
- OWASP Cheat Sheet Series
- Mozilla Web Security
- Application Security Wiki
- OWASP ASVS 4.0
- The System Design Primer
- awesome-scalability
- Web Architecture 101

HTTP Basics

Features and architecture
HTTP/2
- How to debug HTTP/2?
HTTP/3
URI vs URL
Connection vs request
HTTP Headers
- Header compression
HTTP Methods
Request
- Request line
  - Methods
  - Request URI
  - HTTP version
- Request header fields
- Message body
- Generate requests
Response
- Status line
  - HTTP version
  - Status codes and reason phrase
- Response header fields
- Message body
HTTP client
- IP address shortcuts
Back-End web architecture
Useful video resources

SSL/TLS Basics

TLS versions
TLS handshake
- In which layer is TLS situated within the TCP/IP stack?
RSA and ECC keys/certificates
Cipher suites
- Authenticated encryption (AEAD) cipher suites
- Why cipher suites are important?
- NGINX and TLS 1.3 Cipher Suites
Diffie-Hellman key exchange
Certificates
- Chain of Trust
  - What is the main purpose of the Intermediate CA?
- Single-domain
- Multi-domain
- Wildcard
- Wildcard SSL doesn't handle root domain?
TLS Server Name Indication
Verify your SSL, TLS & Ciphers implementation
Useful video resources

NGINX Basics

Processes
- CPU pinning
- Shutdown of worker processes
Configuration syntax
- Comments
- End of lines
- Variables, Strings, and Quotes
- Directives, Blocks, and Contexts
- External files
- Measurement units
- Regular expressions with PCRE
- Enable syntax highlighting
Connection processing
- Event-Driven architecture
- Multiple processes
- Simultaneous connections
- HTTP Keep-Alive connections
- sendfile, tcp_nodelay, and tcp_nopush
Server blocks logic
- Matching location
  - if in location
  - Nested locations
- rewrite vs return
- try_files directive
- if, break and set
- root vs alias
- internal directive
- External and internal redirects
- allow and deny
- uri vs request_uri
Compression and decompression
- What is the best NGINX compression gzip level?
Hash tables
- Server names hash table
Log files
- Conditional logging
- Manually log rotation
- NGINX upstream variables returns 2 values
Reverse proxy
- Passing requests
- Trailing slashes
- Processing headers
- Passing headers
  - Importance of the Host header
  - Redirects and X-Forwarded-Proto
  - A warning about the X-Forwarded-For
  - Improve extensibility with Forwarded
- Response headers
Load balancing algorithms
- Backend parameters
- Upstream servers with SSL
- Round Robin
- Weighted Round Robin
- Least Connections
- Weighted Least Connections
- IP Hash
- Generic Hash
- Fair module
- Other methods
Rate Limiting
- Variables
- Directives, keys, and zones
- Burst and nodelay parameters
NAXSI Web Application Firewall
OWASP ModSecurity Core Rule Set (CRS)
Other subjects
- Secure Distribution of SSL Private Keys with NGINX
Core modules
- ngx_http_geo_module
3rd party modules
- ngx_set_misc
- ngx_http_geoip_module

Helpers

Installing from source
- Automatic installation on RHEL/Debian/BSD
- Compiler and linker
  - Debugging Symbols
- SystemTap
  - stapxx
- Separation and improvement of installation methods
- Installation Nginx on CentOS 7
- Installation OpenResty on CentOS 7
- Installation Tengine on Ubuntu 18.04
- Installation Nginx on FreeBSD 11.3
- Installation Nginx on FreeBSD 11.3 (from ports)
Monitoring
- CollectD, Prometheus, and Grafana
  - nginx-vts-exporter
- CollectD, InfluxDB, and Grafana
- Telegraf, InfluxDB, and Grafana
Testing
- Build OpenSSL 1.0.2-chacha version
- Send request and show response headers
- Send request with http method, user-agent, follow redirects and show response headers
- Send multiple requests
- Testing SSL connection
- Testing SSL connection (debug mode)
- Testing SSL connection with SNI support
- Testing SSL connection with specific SSL version
- Testing SSL connection with specific cipher
- Verify 0-RTT
- Testing SCSV
- Load testing with ApacheBench (ab)
  - Standard test
  - Test with Keep-Alive header
- Load testing with wrk2
  - Standard scenarios
  - POST call (with Lua)
  - Random paths (with Lua)
  - Multiple paths (with Lua)
  - Random server address to each thread (with Lua)
  - Multiple json requests (with Lua)
  - Debug mode (with Lua)
  - Analyse data pass to and from the threads
  - Parsing wrk result and generate report
- Load testing with locust
  - Multiple paths
  - Multiple paths with different user sessions
- TCP SYN flood Denial of Service attack
- HTTP Denial of Service attack
Debugging
- Show information about processes
- Check memory usage
- Show open files
- Check segmentation fault messages
- Dump configuration
- Get the list of configure arguments
- Check if the module has been compiled
- Show the most accessed IP addresses (ip and url)
- Show the most requested urls with http methods
- Show the most accessed response codes
- Calculating requests per second with IP addresses and urls
- Check that the gzip_static module is working
- Which worker processing current request
- Capture only http packets
- Extract User Agent from the http packets
- Capture only http GET and POST packets
- Capture requests and filter by source ip and destination port
- Capture HTTP requests/responses in real time, filter by GET, HEAD and save to a file
- Server Side Include (SSI) debugging
- Dump a process's memory
- GNU Debugger (gdb)
  - Dump configuration from a running process
  - Show debug log in memory
  - Core dump backtrace
- Debugging socket leaks
- SystemTap cheatsheet
  - stapxx
Errors & Issues
- Common errors
Configuration snippets
- Nginx server header removal
- Custom log formats
- Log only 4xx/5xx
- Restricting access with client certificate
- Restricting access by geographical location
  - GeoIP 2 database
- Custom error pages
- Dynamic error pages with SSI
- Limiting the rate of requests per IP with geo and map
- Using trailing slashes
- Properly redirect all HTTP requests to HTTPS
- Adding and removing the www prefix
- Proxy/rewrite and keep the original URL
- Proxy/rewrite and keep the part of original URL
- Proxy/rewrite without changing the original URL (in browser)
- Modify 301/302 response body
- Redirect POST request with payload to external endpoint
- Route to different backends based on HTTP method
- Redirect users with certain IP to special location
- Allow multiple cross-domains using the CORS headers
- Set correct scheme passed in X-Forwarded-Proto
- Securing URLs with the Secure Link Module
- Tips and methods for high load traffic testing (cheatsheet)
- Location matching examples
- Passing requests to the backend
  - The HTTP backend server
  - The uWSGI backend server
  - The FastCGI backend server
  - The memcached backend server
  - The Redis backend server
- HTTPS traffic to upstream servers
- TCP and UDP load balancing
- Lua snippets
- nginscripts snippets
Other snippets
- Recreate base directory
- Create a temporary static backend
- Create a temporary static backend with SSL support
- Generate password file with htpasswd command
- Generate private key without passphrase
- Generate private key with passphrase
- Remove passphrase from private key
- Encrypt existing private key with a passphrase
- Generate CSR
- Generate CSR (metadata from existing certificate)
- Generate CSR with -config param
- Generate private key and CSR
- List available EC curves
- Generate ECDSA private key
- Generate private key and CSR (ECC)
- Generate self-signed certificate
- Generate self-signed certificate from existing private key
- Generate self-signed certificate from existing private key and csr
- Generate multidomain certificate (Certbot)
- Generate wildcard certificate (Certbot)
- Generate certificate with 4096 bit private key (Certbot)
- Generate DH public parameters
- Display DH public parameters
- Extract certs from p7b
- Convert DER to PEM
- Convert PEM to DER
- Verification of the certificate's supported purposes
- Verification of the private key
- Check private key
- Get public key from private key
- Verification of the public key
- Verification of the certificate
- Verification of the CSR
- Check the private key and the certificate are match
- TLSv1.3 and CCM ciphers

Base Rules

Format, prettify and indent your Nginx code
Never use a hostname in a listen or upstream directives
Set the HTTP headers with add_header and proxy*header directives properly
Making a rewrite absolute (with scheme)
Use return directive for URL redirection (301, 302)
Use simple custom error pages
Configure log rotation policy
Don't duplicate index directive, use it only in the http block

Debugging

Improve debugging by disable daemon, master process, and all workers except one
Use core dumps to figure out why NGINX keep crashing
Use mirror module to copy requests to another backend
Dynamic debugging with echo module
Dynamic debugging with SSI

Performance

Enable OCSP Stapling
Avoid multiple index directives
Use $request_uri to avoid using regular expressions
Use try_files directive to ensure a file exists
Don't pass all requests to the backend - use try_files
Use return directive instead of rewrite for redirects
Enable PCRE JIT to speed up processing of regular expressions
Set proxy timeouts for normal load and under heavy load
Configure kernel parameters for high load traffic
Activate the cache for connections to upstream servers

Hardening

Keep NGINX up-to-date
Take care about your ACL rules
Use only the latest supported OpenSSL version
Remove support for legacy and risky HTTP request headers
Prevent Replay Attacks on Zero Round-Trip Time
Prevent caching of sensitive data
Limit concurrent connections
Set properly files and directories permissions (also with acls) on a paths
Implement HTTPOnly and secure attributes on cookies

Reverse Proxy

Use pass directive compatible with backend protocol
Be careful with trailing slashes in proxy_pass directive
Set and pass Host header only with $host variable
Set properly values of the X-Forwarded-For header
Don't use X-Forwarded-Proto with $scheme behind reverse proxy
Always pass Host, X-Real-IP, and X-Forwarded headers to the backend
Use custom headers without X- prefix
Always use $request_uri instead of $uri in proxy_pass
Set proxy buffers and timeouts

Others

Set the certificate chain correctly
Define security policies with security.txt
Use tcpdump to diagnose and troubleshoot the HTTP issues

If you have any idea, send it back to me or add a pull request.

RSS Feed & Updates

GitHub exposes an RSS/Atom feed of the commits, which may also be useful if you want to be kept informed about all changes.

Checklist to rule them all

This checklist was the primary aim of the nginx-admins-handbook. It contains a set of best practices and recommendations on how to configure and maintain the NGINX properly.

This checklist contains all rules (79) from this handbook.

Generally, I think that each of these principles is important and should be considered. I separated them into four levels of priority to help guide your decision.

NAME	AMOUNT	DESCRIPTION
critical	33	definitely use this rule, otherwise it will introduce high risks of your NGINX security, performance, and other
major	26	it's also very important but not critical, and should still be addressed at the earliest possible opportunity
normal	12	there is no need to implement but it is worth considering because it can improve the NGINX working and functions
minor	8	as an option to implement or use (not required)

Remember, these are only guidelines. My point of view may be different from yours so if you feel these priority levels do not reflect your configurations commitment to security, performance or whatever else, you should adjust them as you see fit.

RULE	CHAPTER	PRIORITY
Define the listen directives with address:port pair ^{Prevents soft mistakes which may be difficult to debug.}	Base Rules
Prevent processing requests with undefined server names ^{It protects against configuration errors, e.g. traffic forwarding to incorrect backends.}	Base Rules
Never use a hostname in a listen or upstream directives ^{While this may work, it will comes with a large number of issues.}	Base Rules
Set the HTTP headers with add_header and proxy_*_header directives properly ^{Set the right security headers for all contexts.}	Base Rules
Configure log rotation policy ^{Save yourself trouble with your web server: configure appropriate logging policy.}	Base Rules
Use simple custom error pages ^{Default error pages reveals information which leads to information leakage vulnerability.}	Base Rules
Use HTTP/2 ^{HTTP/2 will make our applications faster, simpler, and more robust.}	Performance
Always keep NGINX up-to-date ^{Use newest NGINX package to fix vulnerabilities, bugs, and to use new features.}	Hardening
Run as an unprivileged user ^{Use the principle of least privilege. This way only master process runs as root.}	Hardening
Protect sensitive resources ^{Hidden directories and files should never be web accessible.}	Hardening
Take care about your ACL rules ^{Test your access-control lists and to stay secure.}	Hardening
Hide upstream proxy headers ^{Don't expose what version of software is running on the server.}	Hardening
Remove support for legacy and risky HTTP request headers ^{Supports for the offending headers should be removed.}	Hardening
Force all connections over TLS ^{Protects your website for handle sensitive communications.}	Hardening
Use min. 2048-bit for RSA and 256-bit for ECC ^{2048 bit (RSA) or 256 bit (ECC) keys are sufficient for commercial use.}	Hardening
Keep only TLS 1.3 and TLS 1.2 ^{Use TLS with modern cryptographic algorithms and without protocol weaknesses.}	Hardening
Use only strong ciphers ^{Use only strong and not vulnerable cipher suites.}	Hardening
Use more secure ECDH Curve ^{Use ECDH Curves with according to NIST recommendations.}	Hardening
Use strong Key Exchange with Perfect Forward Secrecy ^{Establishes a shared secret between two parties that can be used for secret communication.}	Hardening
Defend against the BEAST attack ^{The server ciphers should be preferred over the client ciphers.}	Hardening
Enable HTTP Strict Transport Security ^{Tells browsers that it should only be accessed using HTTPS, instead of using HTTP.}	Hardening
Reduce XSS risks (Content-Security-Policy) ^{CSP is best used as defence-in-depth. It reduces the harm that a malicious injection can cause.}	Hardening
Control the behaviour of the Referer header (Referrer-Policy) ^{The default behaviour of referrer leaking puts websites at risk of privacy and security breaches.}	Hardening
Provide clickjacking protection (X-Frame-Options) ^{Defends against clickjacking attack.}	Hardening
Prevent some categories of XSS attacks (X-XSS-Protection) ^{Prevents to render pages if a potential XSS reflection attack is detected.}	Hardening
Prevent Sniff Mimetype middleware (X-Content-Type-Options) ^{Tells browsers not to sniff MIME types.}	Hardening
Reject unsafe HTTP methods ^{Only allow the HTTP methods for which you, in fact, provide services.}	Hardening
Prevent caching of sensitive data ^{It helps to prevent critical data (e.g. credit card details, or username) leaked.}	Hardening
Limit concurrent connections ^{Limit concurrent connections to prevent a rogue guys from repeatedly connecting to and monopolizing NGINX.}	Hardening
Use pass directive compatible with backend protocol ^{Set pass directive only to working with compatible backend layer protocol.}	Reverse Proxy
Set properly values of the X-Forwarded-For header ^{Identify clients communicating with servers located behind the proxy.}	Reverse Proxy
Don't use X-Forwarded-Proto with $scheme behind reverse proxy ^{Prevent pass incorrect value of this header.}	Reverse Proxy
Always use $request_uri instead of $uri in proxy_pass ^{You should always pass unchanged URI to the backend layer.}	Reverse Proxy
Organising Nginx configuration ^{Well organised code is easier to understand and maintain.}	Base Rules
Format, prettify and indent your Nginx code ^{Formatted code is easier to maintain, debug, and can be read and understood in a short amount of time.}	Base Rules
Use reload option to change configurations on the fly ^{Graceful reload of the configuration without stopping the server and dropping any packets.}	Base Rules
Use return directive for URL redirection (301, 302) ^{The by far simplest and fastest because there is no regexp that has to be evaluated.}	Base Rules
Maintaining SSL sessions ^{Improves performance from the clients’ perspective.}	Performance
Enable OCSP Stapling ^{Enable to reduce the cost of an OCSP validation.}	Performance
Use exact names in a server_name directive if possible ^{Helps speed up searching using exact names.}	Performance
Avoid checks server_name with if directive ^{It decreases NGINX processing requirements.}	Performance
Use $request_uri to avoid using regular expressions ^{By default, the regex is costly and will slow down the performance.}	Performance
Use try_files directive to ensure a file exists ^{Use it if you need to search for a file, it saving duplication of code also.}	Performance
Use return directive instead of rewrite for redirects ^{Use return directive to more speedy response than rewrite.}	Performance
Enable PCRE JIT to speed up processing of regular expressions ^{NGINX with PCRE JIT is much faster than without it.}	Performance
Activate the cache for connections to upstream servers ^{Nginx can now reuse its existing connections (keepalive) per upstream.}	Performance
Disable unnecessary modules ^{Limits vulnerabilities, improve performance and memory efficiency.}	Hardening
Hide Nginx version number ^{Don't disclose sensitive information about NGINX.}	Hardening
Hide Nginx server signature ^{Don't disclose sensitive information about NGINX.}	Hardening
Use only the latest supported OpenSSL version ^{Stay protected from SSL security threats and don't miss out of new features.}	Hardening
Prevent Replay Attacks on Zero Round-Trip Time ^{0-RTT is disabled by default but you should know that enabling this option creates a significant security risks.}	Hardening
Mitigation of CRIME/BREACH attacks ^{Disable HTTP compression or compress only zero sensitive content.}	Hardening
Deny the use of browser features (Feature-Policy) ^{A mechanism to allow and deny the use of browser features.}	Hardening
Control Buffer Overflow attacks ^{Prevents errors are characterised by the overwriting of memory fragments of the NGINX process.}	Hardening
Mitigating Slow HTTP DoS attacks (Closing Slow Connections) ^{Prevents attacks in which the attacker sends HTTP requests in pieces slowly.}	Hardening
Set and pass Host header only with $host variable ^{Use of the $host is the only one guaranteed to have something sensible.}	Reverse Proxy
Always pass Host, X-Real-IP, and X-Forwarded headers to the backend ^{It gives you more control of forwarded headers.}	Reverse Proxy
Set the certificate chain correctly ^{Send the complete chain to the client.}	Others
Enable DNS CAA Policy ^{Allows domain name holders to indicate to CA whether they are authorized to issue digital certificates.}	Others
Separate listen directives for 80 and 443 ports ^{Help you maintain and modify your configuration.}	Base Rules
Use only one SSL config for the listen directive ^{Prevents multiple configurations on the same listening address.}	Base Rules
Use geo/map modules instead of allow/deny ^{Provides the perfect way to block invalid visitors.}	Base Rules
Set global root directory for unmatched locations ^{Specifies the root directory for an undefined locations.}	Base Rules
Don't duplicate index directive, use it only in the http block ^{Watch out for duplicating the same rules.}	Base Rules
Adjust worker processes ^{You can adjust this value to maximum throughput under high concurrency.}	Performance
Make an exact location match to speed up the selection process ^{Exact location matches are often used to speed up the selection process.}	Performance
Use limit_conn to improve limiting the download speed ^{Limits NGINX download speed per connection.}	Performance
Be careful with trailing slashes in proxy_pass directive ^{Incorrect setting could end up with some strange url.}	Reverse Proxy
Use custom headers without X- prefix ^{The use of custom headers with X- prefix is discouraged.}	Reverse Proxy
Tweak passive health checks ^{Improve behaviour of the passive health checks.}	Load Balancing
Define security policies with security.txt ^{Helps make things easier for companies and security researchers.}	Others
Map all the things... ^{Map module provides a more elegant solution for clearly parsing a big list of regexes.}	Base Rules
Use custom log formats ^{This is extremely helpful for debugging specific location directives.}	Debugging
Use debug mode to track down unexpected behaviour ^{There's probably more detail than you want, but that can sometimes be a lifesaver.}	Debugging
Improve debugging by disable daemon, master process, and all workers except one ^{This simplifies the debugging and lets test configurations rapidly.}	Debugging
Use core dumps to figure out why NGINX keep crashing ^{Enable core dumps when your NGINX instance receive an unexpected error or when it crashed.}	Debugging
Use mirror module to copy requests to another backend ^{Use mirroring for investigation and debugging of any original request.}	Debugging
Don't disable backends by comments, use down parameter ^{Is a good solution to marks the server as permanently unavailable.}	Load Balancing
Use tcpdump to diagnose and troubleshoot the HTTP issues ^{Use tcpdump to monitor HTTP.}	Others

Bonus Stuff

You can find here a few of the different things I've worked and included to this repository. I hope that these extras will be useful.

Configuration reports

Many of these recipes have been applied to the configuration of my old private website.

An example configuration is in the configuration examples chapter. It's also based on this version of printable high-res hardening cheatsheets.

SSL Labs

Read about SSL Labs grading here (SSL Labs Grading 2018).

Short SSL Labs grades explanation:

A+ is clearly the desired grade, both A and B grades are acceptable and result in adequate commercial security. The B grade, in particular, may be applied to configurations designed to support very wide audiences (for old clients).

I finally got A+ grade and following scores:

Certificate = 100%
Protocol Support = 100%
Key Exchange = 90%
Cipher Strength = 90%

Look also at the following recommendations. I believe the right configuration of NGINX should give the following SSL Labs scores and provides the best security for the most cases:

Recommended
- A/A+
- Certificate: 100/100
- Protocol Support: 95/100
- Key Exchange: 90/100
- Cipher Strength: 90/100
Perfect but restrictive
- A+
- Certificate: 100/100
- Protocol Support: 100/100
- Key Exchange: 100/100
- Cipher Strength: 100/100

Something about SSL Labs grading mechanism (that's an interesting point of view):

The whole grading mechanism is more propaganda and public relations than actual security. If you want good security, then you must mind the details and understand how things work internally. If you want a good grade then you should do whatever it takes to have a good grade. An "A+" from SSL Labs is a very nifty thing to add at the end of a report, but it does not really equate with having rock solid security. Having an "A+" equates with being able to say "I have an A+". - from this answer by Tom Leek.

Mozilla Observatory

Read about Mozilla Observatory here and about Observatory Scoring Methodology.

I also got the highest summary note (A+) on the Observatory with a very high test score (120/100, max. 135/100):

Printable hardening cheatsheets

I created two versions of printable posters with hardening cheatsheets (High-Res 5000x8800) based on recipes from this handbook:

For xcf and pdf formats please see this directory.

A+ with all 100%’s on @ssllabs and 120/100 on @mozilla observatory:

It provides the highest scores of the SSL Labs test. Setup is very restrictive with 4096-bit private key, only TLS 1.2, and also modern strict TLS cipher suites (non 128-bits). Think carefully about its use (no TLS 1.3, restrictive cipher suites), in my opinion, it is only suitable for obtaining the highest possible rating and seems a little impractical.

A+ on @ssllabs and 120/100 on @mozilla observatory with TLS 1.3 support:

It provides less restrictive setup with 2048-bit key for RSA or 256-bit key for ECC, TLS 1.3 and 1.2, modern strict TLS cipher suites (128/256-bits), and 2048-bit predefined DH groups recommended by Mozilla. The final grade is also in line with the industry standards and guidance. Recommend using this, for me, it is very reasonable configuration.

Fully automatic installation

I created a set of scripts for unattended installation of NGINX from the raw, uncompiled code. It allows you to easily install, create a setup for dependencies (like zlib or openssl), and customized with installation parameters.

For more information please see Installing from source - Automatic installation chapter which describes the installation of NGINX on systems/distros such as Ubuntu, Debian, CentOS, and FreeBSD.

Static error pages generator

I created a simple to use generator for static pages to replace the default error pages that comes with any web server like NGINX.

For more information please see HTTP Static Error Pages Generator.

Server names parser

I added scripts for fast multiple domain searching in the configuration. These tools get specific server_name matches and print them on the screen as a server { ... } blocks. Both are very helpful if you really have tons of domains or if you want to list specific vhosts from file or the active configuration.

You must follow one important rule to be able to use it. Your server block must have the following structure:

server {

  server_name example.com example.org;

  ... # other directives

}

Example of use:

./snippets/server-name-parser/check-server-name.sh example.com
Searching 'example.com' in '/usr/local/etc/nginx' (from disk)

/usr/local/etc/nginx/domains/example.com/servers.conf:79: return 301 https://example.com$request_uri;
/usr/local/etc/nginx/domains/example.com/servers.conf:252: return 301 https://example.com$request_uri;
/usr/local/etc/nginx/domains/example.com/servers.conf:3825: server_name example.com;

Searching 'example.com' in server contexts (from a running process)

>>>>>>>>>> BEG >>>>>>>>>>
server {

  include listen/192.168.252.10/https.example.com.conf;

  server_name example.com;

  location / {

    return 204 "RFC 792";

  }

  access_log /var/log/nginx/example.com/access.log standard;
  error_log /var/log/nginx/example.com/error.log warn;

}
<<<<<<<<<< END <<<<<<<<<<

For more information please see snippets/server-name-parser directory.

Books