{"id":224,"date":"2016-12-01T11:22:21","date_gmt":"2016-12-01T10:22:21","guid":{"rendered":"http:\/\/blog.le-vert.net\/?p=224"},"modified":"2016-12-02T08:19:09","modified_gmt":"2016-12-02T07:19:09","slug":"nginx-ssl-vhosting-using-server-name-indication","status":"publish","type":"post","link":"https:\/\/blog.le-vert.net\/?p=224","title":{"rendered":"Nginx SSL vhosting using Server Name Indication"},"content":{"rendered":"
\n\t\t\t\t\tTweet<\/a>\n\t\t\t\t<\/div><\/div>

Here is the issue: I have a tcp\/443 DNAT to a specific machine running some specific HTTPS app that does not work behind a reverse proxy.<\/p>\n

Obviously, I want to run others application on 443 and I’m not allowed to get any other port.<\/p>\n

Sounds pretty bad, right ?
\nActually, there’s a way out and it’s called “nginx-is-so-fuckin-powerfull” \ud83d\ude09<\/p>\n

As you may know, a long time ago a feature has been added to TLS which is called “Server Name Indication”. Before this it was impossible to serve multiple virtual hosts on a single address because SSL session was negociated before the client actually sends the requested vhost name.<\/p>\n

With SNI, there’s a quick chat between your HTTPS server and the remote browser, something like:<\/p>\n

\r\n- Client: hey I'm an HTTPS client\r\n- Server: Ok, which server ?\r\n- Client: blog.le-vert.net\r\n- Server: Serving blog.le-vert.net certificate...\r\n- Client: #*\/-[}$$ (start talking SSL)\r\n<\/pre>\n
Ok that’s probably not really accurate but who cares about what exactly happens. The thing is: there’s a routing capability before serving the SSL certificate and we know the requested domain name at this point; and guess what: NGINX offers routing possibility using SNI name !!<\/p>\n
First thing… You need a really really new NGINX version (1.11.5), but if your distro doesn’t have it you can use NGINX repositories.
\nSecond, you must understand that very old clients may not use SNI. If it doesn’t it will hit the NGINX default vhost. So make sure to keep the old behavior as default, just in case.
\nHere is the client compatibility list for SNI: https:\/\/en.wikipedia.org\/wiki\/Server_Name_Indication
\nI leave it to you to decide if you care about handling Internet Explorer < 7.\n\nSo let's configure NGINX correctly:\n\nYou need to define a stream {} section on nginx.conf top, just like the http one.\n\n\n
\r\nstream {\r\n    include \/etc\/nginx\/stream.conf.d\/*.conf;\r\n}\r\n<\/pre>\n
Of course, you need to disable default http\/server to listen on port 443 (comment lines like “listen 443 ssl” in all your existing configuration).<\/p>\n
Now we’ll create a stream server, which is a plain TCP proxy:
\nIn \/etc\/nginx\/stream.conf.d\/443.conf:<\/p>\n
\r\nmap $ssl_preread_server_name $name {\r\n    default original_dest;\r\n    new.hostname.com local_https;\r\n}\r\n\r\nupstream original_dest {\r\n    server 1.2.3.4:443;\r\n}\r\n\r\nupstream local_https {\r\n    server 127.0.0.1:8443;\r\n}\r\n\r\nlog_format stream_routing '$remote_addr [$time_local] '\r\n                          'with SNI name \"$ssl_preread_server_name\" '\r\n                          'proxying to \"$name\" '\r\n                          '$protocol $status $bytes_sent $bytes_received '\r\n                          '$session_time';\r\n\r\nserver {\r\n    listen 443;\r\n    ssl_preread on;\r\n    proxy_pass $name;\r\n    access_log \/var\/log\/nginx\/stream_443.log stream_routing;\r\n}<\/pre>\n
And that’s it \ud83d\ude00<\/p>\n
You can now create a new http\/server instance on port 8443 to serve your different new https vhosts but I suggest starting with the default virtual host (\/etc\/nginx\/conf.d\/default.conf) by adding “listen 8443 ssl default_server” and some ssl cert and key directives.<\/p>\n
Here is a example of the stream_443.log:<\/p>\n
\r\n192.168.0.100 [01\/Dec\/2016:11:16:53 +0100] with SNI name \"\" proxying to \"original_dest\" TCP 200 3135 1161 10.256\r\n192.168.0.100 [01\/Dec\/2016:11:17:56 +0100] with SNI name \"new.hostname.com\" proxying to \"local_https\" TCP 200 1467 747 0.070\r\n192.168.0.100 [01\/Dec\/2016:11:18:12 +0100] with SNI name \"new.hostname.com\" proxying to \"local_https\" TCP 200 16505 1365 16.178\r\n192.168.0.100 [01\/Dec\/2016:11:18:15 +0100] with SNI name \"local.server.hostname\" proxying to \"original_dest\" TCP 200 2461 557 25.59\r\n<\/pre>\n
Nice work NGINX, as usual !<\/p>\n
Going further:
\nThere’s just a little issue here: The real HTTPS on port 8443 will always see incoming IP address as 127.0.0.1. Howerver, there’s an overhead called “proxy_protocol” that can help passing proxying related things between NGINX servers but my equipment running behind doesn’t like this.<\/p>\n
So the idea here is to use proxy_protocol between my stream\/443 and http\/8443 instances and strip it when proxying to original_dest using a dummy stream server that does nothing else that popping out the proxy_protocol data and forwarding to the real server. Then I will restore remote_addr in http\/8443.<\/p>\n
The new config file is now:<\/p>\n
\r\nmap $ssl_preread_server_name $name {\r\n    default original_dest;\r\n    new.hostname.com local_https;\r\n}\r\n\r\nupstream original_dest {\r\n    # Forward to a dummy server to strip out proxy_protocol\r\n    # Otherwise original_dest won't work\r\n    server 127.0.0.1:8080;\r\n}\r\n\r\nupstream local_https {\r\n    server 127.0.0.1:8443;\r\n}\r\n\r\nlog_format stream_routing '$remote_addr [$time_local] '\r\n                          'with SNI name \"$ssl_preread_server_name\" '\r\n                          'proxying to \"$name\" '\r\n                          '$protocol $status $bytes_sent $bytes_received '\r\n                          '$session_time';\r\nserver {\r\n    listen 443;\r\n    ssl_preread on;\r\n    proxy_pass $name;\r\n    proxy_protocol on;\r\n    access_log \/var\/log\/nginx\/stream_443.log stream_routing;\r\n}\r\n\r\n# Dummy server to strip out proxy_protocol before sending to original_dest\r\nserver {\r\n    listen 8080 proxy_protocol ;\r\n    proxy_pass 1.2.3.4:443;\r\n}\r\n<\/pre>\n
In the http\/8443 vhost, we set the following to restore original client IP address:<\/p>\n
\r\nlisten 8443 default_server proxy_protocol ssl;\r\nset_real_ip_from 127.0.0.1\/32;\r\nreal_ip_header proxy_protocol;\r\n<\/pre>\n
Nginx -_-<\/p>\n
Bonus stuff:<\/p>\n
I case you’re having issue with SELinux (and you will, for instance it will deny NGINX to start a connection from port 8080 to a remote host), you can use the following to extract failures from audit.log and turn them into a permanent SELinux exception<\/p>\n
\r\ntail -n 2 \/var\/log\/audit\/audit.log (you may want to get more or less lines, depending of what you see happening)\r\ntail -n 2 \/var\/log\/audit\/audit.log |audit2allow -m nginx_proxy_connect (create a plain text SELinux rule, so you can see what's going to be done)\r\ntail -n 2 \/var\/log\/audit\/audit.log |audit2allow -M nginx_proxy_connect (create the real SELinux rule)\r\nsemodule -i nginx_proxy_connect.pp (install the rule)\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"
Here is the issue: I have a tcp\/443 DNAT to a specific machine running some specific HTTPS app that does not work behind a reverse proxy. Obviously, I want to run others application on 443 and I’m not allowed to … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=\/wp\/v2\/posts\/224"}],"collection":[{"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=224"}],"version-history":[{"count":11,"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=\/wp\/v2\/posts\/224\/revisions"}],"predecessor-version":[{"id":235,"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=\/wp\/v2\/posts\/224\/revisions\/235"}],"wp:attachment":[{"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=224"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=224"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.le-vert.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=224"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}