Nginx

How to Configure Custom Access and Error Log Formats in Nginx

How to Configure Custom Access and Error Log Formats in Nginx &-8211; this Article or News was published on this date:2019-05-28 16:38:05 kindly share it with friends if you find it helpful

Nginx HTTP server has a phenomenal logging facility which is highly customizable. In this article, we will explain how to configure you own formats for access and error logs for Nginx in Linux.

The aim of this guide is to help you understand how logs are generated, so as to configure custom log formats for purposes of debugging, troubleshooting or analysis of what unfolds within your web server as well as web applications (such as tracing requests).

Read Also: 4 Good Open Source Log Monitoring and Management Tools for Linux

This article is made of three sections which will enlighten you about configuring access/error logs and how to enable conditional logging in Nginx.

Configuring Access Logs in Nginx

Under Nginx, all client requests to the server are recored in the access log in a specified format using the ngx_http_log_module module.

The default log file is log/access.log (usually /var/log/nginx/access_log on Linux systems) and the default format for logging is normally the combined or main format (this can vary from one distro to another).

The access_log directive (applicable in the http, server, location, if in location and limit except context) is used to set the log file and the log_format directive (applicable under the http context only) is used to set the log format. The log format is described by common variables, and variables that generated only at the time when a log is written.

The syntax for configuring a log format is:

log_format format_name 'set_of_variables_to_define_format';

and the syntax for configuring access log is:

access_log /path/to/log_file format_name;		-simplest form 
OR
access_log /path/to/log_file [format [buffer=size] [gzip[=level]] [flush=time] [if=condition]];

The following is a excerpt from the default Nginx configuration file /etc/nginx/nginx.conf on CentOS 7.

/etc/nginx/nginx.conf
http {
	-main log format 
	log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                               '$status $body_bytes_sent "$http_referer" '
                               '"$http_user_agent" "$http_x_forwarded_for"';

	access_log /var/log/nginx/access.log;
}

This log format yields the following log entry.

127.0.0.1 - dbmanager [20/Nov/2017:18:52:17 +0000] "GET / HTTP/1.1" 401 188 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0"

The following is another useful logging format which we use for tracing requests to our web applications using the some of the default variables, it most importantly has the request ID and logs client location details (country, country code, region and city).

/etc/nginx/nginx.conf
log_format  custom '$remote_addr - $remote_user [$time_local] '
                         	     '"$request" $status $body_bytes_sent '
                      		     '"$http_referer" "$http_user_agent" '
                     		     '"$http_x_forwarded_for" $request_id '
                   		     '$geoip_country_name $geoip_country_code '
                  		     '$geoip_region_name $geoip_city ';

You can use it like this:

access_log  /var/log/nginx/access.log custom;

This will produce a log entry which appears like this.

153.78.107.192 - - [21/Nov/2017:08:45:45 +0000] "POST /ngx_pagespeed_beacon?url=https%3A%2F%2Fwww.example.com%2Fads%2Ffresh-oranges-1509260795 HTTP/2.0" 204 0 "https://www.suasell.com/ads/fresh-oranges-1509260795" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0" "-" a02b2dea9cf06344a25611c1d7ad72db Uganda UG Kampala Kampala 

You can specify several logs using the access_log directives on the same level, here we are using more than one log file in the http context.

/etc/nginx/nginx.conf
http{
	--default log format
	log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                                	      '$status $body_bytes_sent "$http_referer" '
                                         '"$http_user_agent" "$http_x_forwarded_for"';
      
	--request tracing using custom format
	log_format custom '$remote_addr - $remote_user [$time_local] '
                                           '"$request" $status $body_bytes_sent '
                                           '"$http_referer" "$http_user_agent" '
                                           '"$http_x_forwarded_for" $request_id '
                                           '$geoip_country_name $geoip_country_code '
                                          '$geoip_region_name $geoip_city ';

	--this uses the default log format
	access_log /var/log/nginx/access.log;

	--this uses the our custom log format
	access_log /var/log/nginx/custom_log custom;
}

The following are more advanced logging configurations examples, which are useful for log formats that contain compression-related variables and for creating compressed log files:

access_log /var/log/nginx/custom_log custom buffer 32k;
access_log /path/to/log.gz compression  gzip  flush=5m;

Configuring Error Logs in Nginx

In case Nginx experiences any glitches, it records information concerning them in the error log. These issues fall under different severity levels: debug, info, notice, warn, error (this is the default level and works globally), crit, alert, or emerg.

The default log file is log/error.log, but it is normally located in /var/log/nginx/ on Linux distributions. The error_log directive is used to specify the log file, and it can be used in the main, http, mail, stream, server, location context (in that order).

You should also note that:

  • Configurations in the main context are always inherited by lower levels in the order above.
  • and configurations in the lower levels override the configurations inherited from the higher levels.

You can configure error logging using the following syntax:

error_log /path/to/log_file log_level;

For example:

error_log /var/log/nginx/error_log warn; 

This will instruct Nginx to log all messages of type warn and more severe log level crit, alert, and emerg messages.

In the next example, messages of crit, alert, and emerg levels will be logged.

error_log /var/www/example1.com/log/error_log crit;

Consider the configuration below, here, we have defined error logging on different levels (in the http and server context). In case of an error, the message is written to only one error log, the one closest to the level where the error has appeared.

/etc/nginx/nginx.conf
http {
	log_format compression '$remote_addr - $remote_user [$time_local] '
                           '"$request" $status $body_bytes_sent '
                           '"$http_referer" "$http_user_agent" "$gzip_ratio"';
	
	error_log  /var/log/nginx/error_log  crit;

    	server {
		listen 80;
		server_name example1.com;

		-this logs errors messages for example1.com only
      		error_log  /var/log/nginx/example1.error_log  warn;
            	…...
	}

     	server {
		listen 80;
		server_name  example2.com;

		-this logs errors messages for example2.com only
        		error_log  /var/log/nginx/example1.error_log;
        		…….
    	}
}

If you use more than one error_log directives as in the configuration below (same level), the messages are written to all specified logs.

/etc/nginx/nginx.conf
server {
		listen 80;
		server_name example1.com;

      		error_log  /var/www/example1.com/log/error_log  warn;
		error_log  /var/log/nginx/example1.error_log  crit;
            	…...
	}

Configuring Conditional Logging in Nginx

In some cases, we may want Nginx to perform conditional logging of messages. Not every message has to be logged by Nginx, therefore we can ignore insignificant or less important log entries from our access logs for particular instances.

We can use the ngx_http_map_module module which creates variables whose values depend on values of other variables. The parameters inside a map block (which should exist in the http content only) specify a mapping between source and resulting values.

For this kind of setting, a request will not be logged if the condition evaluates to “0” or an empty string. This example excludes requests with HTTP status codes 2xx and 3xx.

/etc/nginx/nginx.conf
http{
	map $status $condition {
		~^[23] 0;
    		default 1;
	}
	server{
		access_log  /path/to/access.log  custom if=$condition;
	}
}

Here is another useful example for debugging a web application in a development phase. This will ignore all messages and only log debug information.

/etc/nginx/nginx.conf
 
http{
	map $info  $debuggable { 
    		default     0; 
    		debug       1; 
	} 
	server{
		……..
		access_log /var/log/nginx/testapp_debug_access_log  debug if=$debuggable; 
		-logs other requests 
		access_log  /var/log/nginx/testapp_access.log  main; 
		…….
	}
}

You can find out more information, including logging to syslog here.

That’s all for now! In this guide, we explained how to configure custom logging format for access and error logs in Nginx. Use the feedback form below to ask questions or share you thoughts about this article.

Amplify – NGINX Monitoring Made Easy

Amplify &-8211; NGINX Monitoring Made Easy &-8211; this Article or News was published on this date:2019-05-28 16:35:12 kindly share it with friends if you find it helpful

Nginx amplify is a collection of useful tools for extensively monitoring a open source Nginx web server and NGINX Plus. With NGINX Amplify you can monitor performance, keep track of systems running Nginx and enables for practically examining and fixing problems associated with running and scaling web applications.

It can be used to visualize and determine a Nginx web server performance bottlenecks, overloaded servers, or potential DDoS attacks; enhance and optimize Nginx performance with intelligent advice and recommendations.

In addition, it can notify you when something is wrong with the any of your application setup, and it also serves as a web application capacity and performance planner.

The Nginx amplify architecture is built on 3 key components, which are described below:

  • NGINX Amplify Backend – the core system component, implemented as a SaaS (Software as a Service). It incorporates scalable metrics collection framework, a database, an analytics engine, and a core API.
  • NGINX Amplify Agent – a Python application which should be installed and run on monitored systems. All communications between the agent and the SaaS backend are done securely over SSL/TLS; all traffic is always initiated by the agent.
  • NGINX Amplify Web UI – a user interface compatible with all major browsers and it is only accessible only via TLS/SSL.

The web UI displays graphs for Nginx and operating system metrics, allows for the creation of a user-defined dashboard, offers a static analyzer to improve Nginx configuration and an alert system with automated notifications.

Step 1: Install Amplify Agent on Linux System

1. Open your web browser, type the address below and create an account. A link will be sent to your email, use it to verify the email address andlogin to your new account.

https://amplify.nginx.com

2. After that, log into your remote server to be monitored, via SSH and download the nginx amplify agent auto-install script using curl or wget command.

$ wget https://github.com/nginxinc/nginx-amplify-agent/raw/master/packages/install.sh
OR
$ curl -L -O https://github.com/nginxinc/nginx-amplify-agent/raw/master/packages/install.sh 

3. Now run the command below with superuser privileges using the sudo command, to install the amplify agent package (the API_KEY will probably be different, unique for every system that you add).

$ sudo API_KEY='e126cf9a5c3b4f89498a4d7e1d7fdccf' sh ./install.sh 
Install Nginx Amplify AgentInstall Nginx Amplify Agent

Install Nginx Amplify Agent

Note: You will possibly get an error indicating that sub_status has not been configured, this will be done in the next step.

4. Once the installation is complete, go back to the web UI and after about 1 minute, you will be able to see the new system in the list on the left.

Step 2: Configure stub_status in NGINX

5. Now, you need to setup stub_status configuration to build key Nginx graphs (Nginx Plus users need to configure either the stub_status module or the extended status module).

Create a new configuration file for stub_status under /etc/nginx/conf.d/.

$ sudo vi /etc/nginx/conf.d/sub_status.conf

Then copy and paste the following stub_status configuration in the file.

server {
    listen 127.0.0.1:80;
    server_name 127.0.0.1;
    location /nginx_status {
        stub_status;
        allow 127.0.0.1;
        deny all;
    }
}

Save and close the file.

6. Next, restart Nginx services to activate the stub_status module configuration, as follows.

$ sudo systemctl restart nginx

Step 3: Configure Additional NGINX Metrics for Monitoring

7. In this step, you need to setup additional Nginx metrics to keep a close eye on your applications performance. The agent will gather metrics from active and growing access.log and error.log files, whose locations it automatically detects. And importantly, it should be allowed to read these files.

All you have to do is define a specific log_format as the one below in your main Nginx configuration file, /etc/nginx/nginx.conf.

log_format main_ext '$remote_addr - $remote_user [$time_local] "$request" '
                                '$status $body_bytes_sent "$http_referer" '
                                '"$http_user_agent" "$http_x_forwarded_for" '
                                '"$host" sn="$server_name" ' 'rt=$request_time '
                                'ua="$upstream_addr" us="$upstream_status" '
                                'ut="$upstream_response_time" ul="$upstream_response_length" '
                                'cs=$upstream_cache_status' ;

Then use the above log format when defining your access_log and the error_log log level should be set to warn as shown.

access_log /var/log/nginx/suasell.com/suasell.com_access_log main_ext;
error_log /var/log/nginx/suasell.com/suasell.com_error_log  warn;

8. Now restart Nginx services once more, to effect the latest changes.

$ sudo systemctl restart nginx

Step 4: Monitor Nginx Web Server Via Amplify Agent

9. Finally, you can begin monitoring your Nginx web server from the Amplify Web UI.

Install Nginx Amplify AgentNginx Amplify Overview

Nginx Amplify Overview

Install Nginx Amplify AgentNginx Amplify Graph

Nginx Amplify Graph

To add a another system to monitor, simply go to Graphs and click on “New System” and follow the steps above.

Nginx Amplify Homepage: https://amplify.nginx.com/signup/

Amplify is a powerful SaaS solution for monitoring your OS, Nginx web server as well as Nginx based applications. It offers a single, unified web UI for keeping an eye on multiple remote systems running Nginx. Use the comment form below to share your thoughts about this tool.

How to Monitor Nginx Performance Using Netdata on CentOS 7

How to Monitor Nginx Performance Using Netdata on CentOS 7 &-8211; this Article or News was published on this date:2019-05-28 16:14:37 kindly share it with friends if you find it helpful

Netdata is a free open source, scalable, adaptive, customizable, extensible, and powerful real-time performance and health monitoring tool for Linux systems, which collects and visualizes metrics. It works on desktops, personal computers, servers, embedded devices, IoT, and more.

Read Also: How to Monitor Apache Performance Using Netdata on CentOS 7

It is a system health monitoring tool which allows you to keep an eye on how your systems and applications or services such as web servers are operating, or why they are slow or misbehaving. It is extremely effective and efficient in terms of CPU usage as well as other system resources.

In this article, we will explain how to monitor Nginx HTTP web server performance using Netdata on CentOS 7 or RHEL 7 distribution.

By the end of this guide, you will be able to watch visualizations of active connections, requests, status, and connection rate of your Nginx web server.

Requirements:

  1. A CentOS 7 Server or RHEL 7 Server with Minimal Install.
  2. Nginx HTTP server installation with ngx_http_stub_status_module enabled.

Step 1: Install Nginx on CentOS 7

1. First start by enabling EPEL Repository and then install Nginx HTTP server from the EPEL software repositories using the YUM package manager.

- yum install epel-release
- yum install nginx 

2. Next, check the version of Nginx installed on your system, it should be compiled with the stub_status module indicated by the --with-http_stub_status_module configuration argument, as shown in the following screenshot.

- nginx -V
Check Nginx VersionCheck Nginx Version

Check Nginx Version

3. After successfully installing Nginx, start it and enable it to auto-start at system boot and ensure that it is up and running.

- systemctl status nginx
- systemctl enable nginx
- systemctl status nginx

4. If you are running firewalld dynamic firewall, you need to open port 80 (HTTP) and 443 (HTTPS) which the web server listens on, for client connection requests.

- firewall-cmd --permanent --add-port=80/tcp
- firewall-cmd --permanent --add-port=443/tcp
- firewall-cmd --reload 

Step 2: Step 2: Enable Nginx Stub_Status Module

5. Now enable stub_status module which netdata uses to collect metrics from your Nginx web server.

- vim /etc/nginx/nginx.conf

Copy and paste the location configuration below into the server block, as shown in the screenshot.

location /stub_status {
 	stub_status;
 	allow 127.0.0.1;	-only allow requests from localhost
 	deny all;		-deny all other hosts	
 }
Check Nginx VersionEnable Nginx Status Module

Enable Nginx Status Module

6. Next, test the new nginx configuration for any errors and restart the nginx service to effect the recent changes.

- nginx -t
- systemctl restart nginx

7. Next, test the nginx status page using the curl command-line tool.

- curl http://127.0.0.1/stub_status
Check Nginx VersionCheck Nginx Status Page

Check Nginx Status Page

Step 3: Install Netdata on CentOS 7

8. There is a one-liner shell script you can use to kick start installation of netdata latest release from its github repository. This script will download another script to detect your Linux distro and installs the required system packages for building netdata; thereafter grabs the latest netdata source files; builds and installs it.

Use the command below to launch the kickstarter script, the option all allows for installing required packages for all netdata plugins including the ones for Nginx.

- bash (curl -Ss https://my-netdata.io/kickstart.sh) all

If your not accessing the system as root, you will be prompted to enter your user password for sudo command, and you will also be asked to confirm certain operations by pressing [Enter].

Check Nginx VersionInstall Netdata on CentOS 7

Install Netdata on CentOS 7

8. After building, and installing netdata, the script will automatically start the netdata service via systemd service manager, and enables it to start at system boot. Netdata listens on port 19999 by default.

Check Nginx VersionNetdata Installation Summary

Netdata Installation Summary

9. Next, open port 19999 in the firewall to access the netdata web UI.

- firewall-cmd --permanent --add-port=19999/tcp
- firewall-cmd --reload 

Step 4: Configure Netdata to Monitor Nginx Performance

9. The netdata configuration for Nginx plugin is stored in the /etc/netdata/python.d/nginx.conf configuration file, written in YaML format.

- vim /etc/netdata/python.d/nginx.conf

The default configuration is enough to get you started with monitoring your Nginx web server.

Check Nginx VersionNetdata Nginx Configuration

Netdata Nginx Configuration

In case you have made any changes to configuration file, after reading the documentation, restart the netdata service to effect the changes.

- systemctl restart netdata

Step 5: Monitor Nginx Performance Using Netdata

10. Now open a web browser and use the following URL to access the netdata web UI.

http://domain_name:19999
OR
http://SERVER_IP:19999

From the plugin list on the right hand side, click on “nginx local” to start monitoring your Nginx web server. You will be able to watch visualizations of active connections, requests, status, and connection rate as shown in the following screenshot.

Check Nginx VersionMonitor Nginx Using Netdata

Monitor Nginx Using Netdata

Netdata Github repository: https://github.com/firehol/netdata

That’s all! Netdata is a real-time, distributed performance and health monitoring tool for Linux systems. In this article, we showed how to monitor Nginx web server performance using netdata on CentOS 7. Use the comment form below to share any queries or thoughts about this guide.