Reverse proxy server snippet provisioning for applications (Connectivity Suite)¶

NOTICE

This feature is only meant to be used as part of the Connectivity Suite Framework and to enable configuration from Common Configurator on IEM.

This guide shows you how to create and use Reverse Proxy Server Snippet for Industrial Edge applications which enables configuration by the Connectivity Suits' Common Configurator.
Industrial Edge Devices utilize NGINX for reverse proxy functionality, with server snippets configured as virtual hosts (vHosts) for gRPC communication between Configurator and runtime on the device.
Virtual hosts cannot be used in parallel with a classic Industrial Edge reverse proxy configuration for the same service/container.

How to create Nginx Server Configuration
How to create an Application with NGINX vHost server Server Snippet

How to create NGINX server configuration¶

In order to create a valid NGINX server configuration, there are requirements and restrictions that must be followed.
These rules are validated and enforced during the application installation. If the configuration does not meet the validation rules, the application installation will be interrupted.

Restrictions for NGINX vHost server configuration:

Only one server block is allowed, from which the server_name should be unique.
Nesting location blocks within other location blocks is not allowed in NGINX configurations.
When forwarding gRPC requests with grpc_pass, the address must be defined in variables, and the variable name should start with grpc_*.
To use location blocks with gRPC, enable gRPC interception of errors by adding the grpc_intercept_errors on; this is a mandatory directive within a location block.

Used directives¶

The types of used directives for the NGINX vHost server configuration can be separated as follows:

Mandatory Directives
Templated Directives
Whitelisted Directives
Location Blocks

Mandatory DirectivesTemplated DirectivesWhitelisted DirectivesLocation Blocks (Mandatory Directives)

The listed directives must always be included in the server configuration, otherwise the app will fail during validation.

Directive	Description
server_name	Specifies the server name; it should match the regex `^[a-z0-9]+(\.[a-z0-9]+)*\.proxy-redirect$`
listen	Defines the port, protocol and security protocol
ssl_certificate	Specifies the path to the SSL certificate file
ssl_certificate_key	Specifies the path to the SSL private key file
ssl_ciphers	Defines the SSL ciphers to be used
ssl_protocols	Specifies the SSL protocols to be used
ssl_ecdh_curve	Specifies the elliptic curve for ECDHE ciphers
ssl_dhparam	Specifies the path to the Diffie-Hellman parameter file
resolver	Specifies the DNS server to use for name resolution
set	Defines variables

NOTICE

set directive is mandatory, since the container address in the location block must use a variable.

Templated directives use pre-defined values as placeholders and which will be rendered internally during the installation. The templated values always match the ones displayed in the table:

Directive	Pre-defined value	Example
listen	{{.ListenGrpc}}	listen 443 ssl http2
ssl_certificate	{{.SslCertPath}}
ssl_certificate_key	{{.SslKeyPath}}
ssl_ciphers	{{.SslCiphers}}
ssl_protocols	{{.SslProtocols}}
ssl_ecdh_curve	{{.SslEcdhCurve}}
ssl_dhparam	{{.SslDhParam}}
resolver	{{.Resolver}}	e.g. 127.0.0.11

NOTICE

The pre-defined values should be enclosed in single quotes in the configuration (see Example).

Whitelisted directives are the only ones which will be accepted for the configuration and are strictly enforced; if a configuration uses one directive not defined in this whitelist, it will not pass the validation, causing the interruption of your app installation.

Directive	Description
server_name	Specifies the server name
listen	Defines the IP address and port to listen on
auth_request	Defines an external authorization request
ssl	Enables SSL/TLS encryption
ssl_*	Prefix for various SSL/TLS related directives
upstream	Defines a group of servers as an upstream group
grpc_pass	Passes requests to a gRPC server
grpc_*	Prefix for various gRPC related directives
proxy_redirect	Defines the redirection behaviour for proxies
client_*	Prefix for various client related directives
allow	Defines an IP address or network to allow access
deny	Defines an IP address or network
default_type	Sets the default content type
add_header	Adds a response header
internal	Restricts access to internal requests
location	Defines a location block
error_page	Defines custom error pages
server	Defines a server block
return	Specifies a response status code
if	Executes directives conditionally
grpc_read_timeout	Sets the timeout for reading from a gRPC server
grpc_send_timeout	Sets the timeout for sending to a gRPC server
grpc_intercept_errors	Enables interception of gRPC errors
proxy_pass	Passes requests to a proxy server
proxy_set_header	Sets headers for requests sent to a proxy server
set	Defines variables
resolver	Specifies the DNS server to use for name resolution

When defining location blocks in the configuration, the following directives should be included.

Directive	Description	Expected values
grpc_pass	Passes requests to a gRPC server	`$grpc_<address-name>`
grpc_intercept_errors	Enables interception of gRPC errors	`on`
auth_request	Defines an external authorization request (Optional)	`{{.AuthEdgeLocation}}` (Optional)

NOTICE

The application's vHost snippet can contain a regular expression (regex) as a location argument. When the application is installed, the regex will not be validated. Please ensure that it is syntactically correct. It is allowed to used a named location as argument for grpc_pass directive. The named locations needs to provided with the vHost configuration and should exists.

Authentication of the requests coming to virtual hosts¶

To add authentication to the requests coming to virtual hosts, you can use the auth_request directive The application developer can follow these guidelines to enable auth in the required location of the application's virtual hosts. At server level, add the following block to inject auth location '{{.AuthEdgeLocationBlock}}'.

And for location blocks use this template:

server {
  set $grpc_address "grpc://<server_name>.proxy-redirect:<port>";
  server_name "<server_name>.proxy-redirect";

  location = '{{.AuthEdgeLocationBlock}}'

  location / {
    auth_request '{{.AuthEdgeLocation}}';
    grpc_pass $grpc_address;
    ...
  }
}

Configuration example¶

In this example, the configuration sets up an NGINX server block with various directives. Here's an overview of the different sections:

The set directive is used to assign values to variables $grpc_address and $grpc_address1.
The resolver directive sets the DNS server to be used for name resolution.
The server_name directive specifies the server name for this block.
The listen directive defines the IP address and port to listen on.
The location blocks define different paths and their corresponding configurations.
The ssl_certificate, ssl_certificate_key, ssl_protocols, ssl_ciphers, ssl_ecdh_curve, and ssl_dhparam directives configure SSL/TLS settings.
The ssl_prefer_server_ciphers, ssl_session_timeout, ssl_session_cache, and ssl_session_tickets directives further configure SSL/TLS.
The error_page directives define mappings for different HTTP and gRPC error codes.
The location blocks with names starting with @ define error responses for specific gRPC error codes.

Expand NGINX vHost server server configuration

server {
    set $grpc_address "grpc://cssvcamdregistry.proxy-redirect:50051";
    resolver 127.0.0.11 ipv6=off;
    #resolver 127.0.0.11 ipv6=off;

    server_name cssvcamdregistry.proxy-redirect;
    listen 443 ssl http2;

    location / {
        auth_request /auth;
        grpc_pass $grpc_address;
        grpc_intercept_errors on;
        grpc_read_timeout 3153600000s;
        grpc_send_timeout 3153600000s;
        client_body_timeout 3153600000s;
        client_max_body_size 0;
    }

    location = /auth {
        internal;
        proxy_pass http://MWAuthEdgeService/a.service/api/v1/auth/token/verify/cookie;
        proxy_pass_request_body off;
        proxy_set_header Content-Length "";
        proxy_set_header X-Real-IP-Edge $remote_addr;
        proxy_set_header X-Auth-Source "IEM";

        error_page 401 = @grpc_unauthenticated;
        error_page 403 = @grpc_permission_denied;
    }

    location @grpc_unauthenticated {
        add_header grpc-status 16 always;  
        add_header grpc-message "unauthenticated" always;  
        add_header Content-Type application/grpc always;  
        return 200;
    }

    location @grpc_permission_denied {
        add_header grpc-status 7 always;
        add_header grpc-message "permission denied" always;
        add_header Content-Type application/grpc always;
        return 200;
    }

    default_type application/grpc;

    ssl_certificate /data/edgecerts/certs/edgepublic.crt;
    ssl_certificate_key /data/edgecerts/private/edgeprivate.key;
    ssl_protocols TLSv1.3;
    ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384;
    ssl_ecdh_curve secp384r1;
    ssl_dhparam /etc/nginx/dhparam.pem;

    ssl_prefer_server_ciphers on;
    ssl_session_timeout  10m;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;

# Standard HTTP-to-gRPC status code mappings
# Ref: https://github.com/grpc/grpc/blob/master/doc/http-grpc-status-mapping.md
#
    error_page 400 = @grpc_internal;
    error_page 401 = @grpc_unauthenticated;
    error_page 403 = @grpc_permission_denied;
    error_page 404 = @grpc_unimplemented;
    error_page 429 = @grpc_unavailable;
    error_page 502 = @grpc_unavailable;
    error_page 503 = @grpc_unavailable;
    error_page 504 = @grpc_unavailable;

# NGINX-to-gRPC status code mappings
# Ref: https://github.com/grpc/grpc/blob/master/doc/statuscodes.md
#
    error_page 405 = @grpc_internal; # Method not allowed
    error_page 408 = @grpc_deadline_exceeded; # Request timeout
    error_page 413 = @grpc_resource_exhausted; # Payload too large
    error_page 414 = @grpc_resource_exhausted; # Request URI too large
    error_page 415 = @grpc_internal; # Unsupported media type;
    error_page 426 = @grpc_internal; # HTTP request was sent to HTTPS port
    error_page 495 = @grpc_unauthenticated; # Client certificate authentication error
    error_page 496 = @grpc_unauthenticated; # Client certificate not presented
    error_page 497 = @grpc_internal; # HTTP request was sent to mutual TLS port
    error_page 500 = @grpc_internal; # Server error
    error_page 501 = @grpc_internal; # Not implemented

# gRPC error responses
# Ref: https://github.com/grpc/grpc-go/blob/master/codes/codes.go
#
    location @grpc_deadline_exceeded {
        add_header grpc-status 4;
        add_header grpc-message 'deadline exceeded';
        return 204;
    }
    location @grpc_permission_denied {
        add_header grpc-status 7;
        add_header grpc-message 'permission denied';
        return 204;
    }
    location @grpc_resource_exhausted {
        add_header grpc-status 8;
        add_header grpc-message 'resource exhausted';
        return 204;
    }
    location @grpc_unimplemented {
        add_header grpc-status 12;
        add_header grpc-message unimplemented;
        return 204;
    }
    location @grpc_internal {
        add_header grpc-status 13;
        add_header grpc-message 'internal error';
        return 204;
    }
    location @grpc_unavailable {
        add_header grpc-status 14;
        add_header grpc-message unavailable;
        return 204;
    }
    location @grpc_unauthenticated {
        add_header grpc-status 16;
        add_header grpc-message unauthenticated;
        return 204;
    }
}

${NGINX_SERVICE_NAME}: The docker DNS name of the container.
${NGINX_SERVICE_PORT}: The grpc tcp port number.

The NGINX vHost server server configuration is templated with following directives and required in same format:

server {  
    set $grpc_address "grpc://cssregdupvhost.proxy-redirect:50051";  
    set $grpc_address1 "grpc://cssregdupvhost1.proxy-redirect:50052";  
    resolver '{{.Resolver}}';  
    #resolver 127.0.0.11 ipv6=off;  

    server_name "cssvcregistry.proxy-redirect";  
    listen '{{.ListenGrpc}}';
}

The server block begins with the declaration of two variables $grpc_address and $grpc_address1 which are assigned the gRPC server addresses to which requests will be forwarded.
The resolver directive is used to specify the DNS server for name resolution. The value is expected to be provided as a variable {{.Resolver}}. Alternatively, the commented out line #resolver 127.0.0.11 ipv6=off; can be used to specify a specific DNS resolver address.
The server_name directive sets the server name to "cssvcregistry.proxy-redirect".
The listen directive defines the IP address and port number on which the server will listen for incoming connections. The value is expected to be provided as a variable {{.ListenGrpc}}.

    location / {  
        grpc_pass $grpc_address;  
        grpc_intercept_errors on;  
        grpc_read_timeout 3153600000s;  
        grpc_send_timeout 3153600000s;  
        client_body_timeout 3153600000s;  
        client_max_body_size 0;  
    }

The location / block defines the configuration for requests to the root path ("/").
The grpc_pass directive specifies that gRPC requests should be forwarded to the address specified in the $grpc_address variable.
The grpc_intercept_errors directive enables the interception of gRPC errors.
Let's break down the remaining part of the template:

location /test/ {  
    auth_request '{{.AuthEdgeLocation}}';  

    grpc_pass $grpc_address1;  
    grpc_intercept_errors on;  
}

The location /test/ block defines the configuration for requests to the "/test/" path. Can be also a regex like ^/test\.testPackage.
The auth_request directive specifies an external authorization request. The value is expected to be provided as a variable {{.AuthEdgeLocation}}.
The grpc_pass directive specifies that gRPC requests should be forwarded to the address specified in the $grpc_address1 variable. The argument can also be a named location.
The grpc_intercept_errors directive enables the interception of gRPC errors for this location.

location = '{{.AuthEdgeLocationBlock}}';  
default_type application/grpc;

The location directive is used to specify an exact match location block with the value provided as a variable {{.AuthEdgeLocationBlock}}.
The default_type directive sets the default content type for responses to "application/grpc".

This template sets up an NGINX server block that listens for gRPC requests on the specified address and port. It forwards requests to the appropriate gRPC server based on the configured locations. It also includes configuration for external authorization requests and handles gRPC error interception.

For SSL/TLS configuration:

ssl_certificate '{{.SslCertPath}}';
This line sets the path to the SSL certificate file. The value is taken from the SslCertPath variable.
ssl_certificate_key '{{.SslKeyPath}}';
Specifies the path to the SSL certificate key file. The value is taken from the SslKeyPath variable.
ssl_protocols '{{.SslProtocols}}';
Sets the SSL/TLS protocols to be used. The value is taken from the SslProtocols variable.
ssl_ciphers '{{.SslCiphers}}';
Specifies the SSL/TLS ciphers to be used for encryption. The value is taken from the SslCiphers variable.
ssl_ecdh_curve '{{.SslEcdhCurve}}';
Sets the elliptic curve for the Diffie-Hellman key exchange. The value is taken from the SslEcdhCurve variable.
ssl_dhparam '{{.SslDhParam}}';
Specifies the path to the Diffie-Hellman parameters file for key exchange. The value is taken from the SslDhParam variable.

These directives are used to configure SSL/TLS settings for the NGINX server. The provided template uses variables ({{.SslCertPath}}, {{.SslKeyPath}}, {{.SslProtocols}}, {{.SslCiphers}}, {{.SslEcdhCurve}}, and {{.SslDhParam}}) to allow dynamic values to be inserted when generating the NGINX configuration file.

How to create an application with NGINX vHost server server snippet¶

NOTICE

Applications with NGINX vHost Server Snippets can not be created directly in the IE App Publisher.

Using IECTL
Manual extension
Backwards Compatibility

Create application with NGINX vHost server snippet using IECTL¶

This method facilitates IECTL for creating the application with NGINX vHost Server Server Snippet.
For the creation of applications, IECTL includes the Publisher plugin. Detailed documentation is available here.

The reverse proxy or NGINX configuration is applied in the version create step for both app-projects and standalone applications.

The vHost Server configration snippet needs to be added through the following flag.

-n, --nginxjson string #JSON map of nginx configuration

It should contain a JSON map with a certain format.

{"<service_name>":[{"vHost":"base64_encoded_vHost_server_snippet"}]}

For two or more services using vHost Server Snippets the following syntax applies:

{
    "<service1_name>": [
        {
            "vHost": "base64_encoded_vHost_server_snippet1"
        }
    ],
    "<service2_name>": [
        {
            "vHost": "base64_encoded_vHost_server_snippet2"
        }
    ],
    "<service_n_name>": [
        {
            "vHost": "base64_encoded_vHost_server_snippet3"
        }
    ]
}

NOTICE

This example nginxjson does not have additional regular reverse proxy configuration.

For example, the IECTL command for a standalone application using the Nginx vHost server snippet might look like this

iectl publisher standalone-app version create \
    -a "my_app" \
    -v "0.0.1" \
    -y "/yaml/docker-compose.yaml" \
    -c "changelogs" \
    -s "" \
    -t "ExternalLink" \
    -u "" \
    -n '{"<service_name>":{"vHost":"base64_encoded_vHost_server_snippet"}}'

Aferwards you can continue exporting or publishing the application version to IEM or IE Hub.

Extend existing Application with NGINX vHost server Server Snippet¶

The application developer must create an application with the desired configuration and functionality.
The application file generated needs to be unzipped.

If nginx folder does not exist it should be added at repository level. Additionally, add the nginx.json file to the nginx folder. For example, the file structure of an application package would look like this:

└── example-app
    └── <app-id_version>
        ├── <repository-name>
        │   ├── attachments
        │   ├── images
        │   ├── nginx
        │   │   ├── nginx.json    
        │   ├── appicon.png
        │   └── docker-compose.yml
        ├── detail.json
        └── digest.json

In nginx.json under the application json object a new key-value pair should be created where the key is called vHost and the value is the reverse proxy server snippet configuration encoded as base64 string.
```
{
  "<service_name>":[
    {
      "vHost": "<encodedstring-base64>"
    }
  ]
}
```
The next steps requires the filehash of the nginx.json which can be obtained like shown below..
```
sha256 reponame/nginx/nginx.json
```

Update or add the file hash of nginx.json file in the digest.json. For example:

{
  "version": "1",
  "files": {
    "detail.json": "47b4db25dacc175061133f400b497a82f066cf60e7081e6dc7aee4bc6f08fe8b",
    "<reponame>/appicon.png": "0143026cc496ab82cbe1b7192c96bf6b2999e3b1dc82e5d49f909ba41362d541",
    "<reponame>/docker-compose.yml": "9f3f3a3ad0df7904e254250ab4b5ff0329f463609b3c4a6826e0dd8519b46044",
    "<reponame>/images/<iamge>.tar": "35666efe4120307c67f956a6a9a0e77de5a37e72c5b585f502ddd642989a637f",
    "<reponame>/nginx/nginx.json": "fddce20d0768e0be31a74ba2aedec91d57519045a881c8ebd3efb598f45f7389"
  }
}

Create an .app file of the application. For that archive to tar like shown below.
```
tar -cvf ../<app-file-name>.app ./*
```

Backwards Compatibility¶

When running the application on older devices (IE Runtime version <1.20) you need to do the following adaptions.

If your application should be backwards compatible you need to provide dummy values to the reverse proxy configuration in the nginx.json file.

You will find an example of this below:

{
    "cssvcregistry": [
        {
            "name": "<dummy>",
            "protocol": "HTTP",
            "port": "<container-port>",
            "headers": "{}",
            "rewriteTarget": "</dummy>",
            "subPath": "",
            "isSecureRedirection": true,
            "bypassUrlDecoding": false,
            "vHost": "<base64-decoded-string>"
        }
    ]
}