Monitoring Tools

How to Monitor Apache Web Server Load and Page Statistics

How to Monitor Apache Web Server Load and Page Statistics &-8211; this Article or News was published on this date:2019-05-28 19:30:42 kindly share it with friends if you find it helpful

As promised in our earlier tutorials of Apache Series, Today we are here with another article on the Apache HTTP web server which can make a System Administrators life much easier to handle load of Apache web server using mod_status module.

What is mod_status?

mod_status is an Apache module which helps to monitor web server load and current httpd connections with an HTML interface which can be accessible via a web browser.

Apache’s mod_status shows a plain HTML page containing the information about current statistics of web server state including.

  1. Total number of incoming requests
  2. Total number of bytes and counts server
  3. CPU usage of Web server
  4. Server Load
  5. Server Uptime
  6. Total Traffic
  7. Total number of idle workers
  8. PIDs with respective client and many more.

The default Apache Project enabled their server statistics page to the general public. To have a demo of busy web site’s status page, visit.

  1. http://www.apache.org/server-status
Testing Environment

We have used following Testing Environment for this article to explore more about mod_status with some practical examples and screen-shots.

  1. Operating System – CentOS 6.5
  2. Application – Apache Web Server
  3. IP Address – 5.175.142.66
  4. DocumentRoot– /var/www/html
  5. Apache Configuration file – /etc/httpd/conf/httpd.conf (for Red Hat)
  6. Default HTTP Port – 80 TCP
  7. Test Configuration Settings – httpd -t

The prerequisites for this tutorial is that you should already aware how to install and configure a Basic Apache Server. If you don’t know how to setup Apache, read the following article that might help you in setting up your own Apache Web Server.

  1. Create Your Own Webserver and Hosting A Website in Linux

How to Enable mod_status in Apache

The default Apache installation comes with mod_status enabled. If not, make sure to enable it in Apache configuration file at.

[[email protected] ~]- vi /etc/httpd/conf/httpd.conf

Search for the word “mod_status” or keep scrolling down until you find a line containing.

-LoadModule status_module modules/mod_status.so

If you see a ‘-‘ character at the beginning of “LoadModule”, that means mod_status is disabled. Remove the ‘&-8211;‘ to enable mod_status.

LoadModule status_module modules/mod_status.so

Configure mod_status

Now again search for the word “Location” or scroll down until you find a section for mod_status which should look like following.

- Allow server status reports generated by mod_status,
- with the URL of http://servername/server-status
- Change the ".example.com" to match your domain to enable.
-
-Location /server-status>
-    SetHandler server-status
-    Order deny,allow
-    Deny from all
-    Allow from .example.com
-/Location>

In the above section, uncomment the lines for Location directive, SetHandler and the directory restrictions according to your needs. For example, I am keeping it simple with the Order Allow, deny and it’s allowed for all.

Location /server-status>
   SetHandler server-status
   Order allow,deny
   Deny from all
   Allow from all 
/Location>

Note : The above configuration is the default configuration for default Apache web site (single website). If you’ve created one or more Apache Virtual Hosts, the above configuration will won’t work.

So, basically, you need to define the same configuration for each virtual host for any domains you’ve configured in Apache. For example, the virtual host configuration for mod_status will look like this.

VirtualHost *:80>
    ServerAdmin [email protected]
    DocumentRoot /var/www/html/example.com
    ServerName example.com
    ErrorLog logs/example.com-error_log
    CustomLog logs/example.com-access_log common
Location /server-status>
   SetHandler server-status
   Order allow,deny
   Deny from all
   Allow from example.com 
/Location>
/VirtualHost>

Enable ExtendedStatus

The “ExtendedStatus” settings adds more information to the statistics page like, CPU usage, request per second, total traffic, etc. To enable it, edit the the same httpd.conf file and search for the word “Extended” and Uncomment the line and set the status “On” for ExtendedStatus directive.

- ExtendedStatus controls whether Apache will generate "full" status
- information (ExtendedStatus On) or just basic information (ExtendedStatus
- Off) when the "server-status" handler is called. The default is Off.
-
ExtendedStatus On

Restart Apache

Now make sure that you’ve correctly enabled and configured Apache server status page. You can also check for the errors in the httpd.conf configuration using following command.

[[email protected] ~]- httpd -t

Syntax OK

Once, you get syntax is OK, you can able to restart the httpd service.

[[email protected] ~]- service httpd restart
Stopping httpd:                                          [  OK  ]
Starting httpd:                                          [  OK  ]

Access mod_status Page

The Apache status page will be accessible via your domain name with “/server-status” at the following URL’s.

http://serveripaddress/server-status

OR

http://serev-hostname/server-status

You will see something similar to the following page with ExtendedStatus enabled.

Apache mod_statusApache mod_status

Apache mod_status View

In above Snapshot, you can see that an HTML interface, which shows all information about server uptime, process Id with its respective client, the page they are trying to access.

It also shows the meaning and usage of all the abbreviations used to display the status which helps us to understand the situation better.

You can also refresh the page every time seconds (say 5 seconds) to see the updated statistics. To set the automate refresh, please add “?refresh=N” at the end of the URL. Where N can be replaced with the number of seconds which you want your page to get refreshed.

http://serveripaddress/server-status/?refresh=5
Apache mod_statusApache mod_status Refresh

Apache mod_status Refresh

Command line Status Page View

You can also view the Apache status page from the command-line interface using the special command-line browsers called links or lynx. You can install them using default package manager utility called yum as shown below.

- yum install links

OR

- yum install lynx

Once, you’ve installed the, you can get the same statistics on your terminal by using following command.

[[email protected] ~]- links http://serveripaddress/server-status
OR
[[email protected] ~]- lynx http://serveripaddress/server-status
OR
[[email protected] ~]-  /etc/init.d/httpd fullstatus
Sample Output
                     Apache Server Status for localhost
   Server Version: Apache/2.2.15 (Unix) DAV/2 PHP/5.3.3
   Server Built: Aug 13 2013 17:29:28

   --------------------------------------------------------------------------
   Current Time: Tuesday, 14-Jan-2014 04:34:13 EST
   Restart Time: Tuesday, 14-Jan-2014 00:33:05 EST
   Parent Server Generation: 0
   Server uptime: 4 hours 1 minute 7 seconds
   Total accesses: 2748 - Total Traffic: 9.6 MB
   CPU Usage: u.9 s1.06 cu0 cs0 - .0135% CPU load
   .19 requests/sec - 695 B/second - 3658 B/request
   1 requests currently being processed, 4 idle workers
 .__.__W...

   Scoreboard Key:
   "_" Waiting for Connection, "S" Starting up, "R" Reading Request,
   "W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
   "C" Closing connection, "L" Logging, "G" Gracefully finishing,
   "I" Idle cleanup of worker, "." Open slot with no current process

Srv PID     Acc    M CPU   SS  Req Conn Child Slot     Client        VHost             Request
0-0 -    0/0/428   . 0.30 5572 0   0.0  0.00  1.34 127.0.0.1      5.175.142.66 OPTIONS * HTTP/1.0
                                                                               GET
1-0 5606 0/639/639 _ 0.46 4    0   0.0  2.18  2.18 115.113.134.14 5.175.142.66 /server-status?refresh=5
                                                                               HTTP/1.1
                                                                               GET
2-0 5607 0/603/603 _ 0.43 0    0   0.0  2.09  2.09 115.113.134.14 5.175.142.66 /server-status?refresh=5
                                                                               HTTP/1.1
3-0 -    0/0/337   . 0.23 5573 0   0.0  0.00  1.09 127.0.0.1      5.175.142.66 OPTIONS * HTTP/1.0
                                                                               GET
4-0 5701 0/317/317 _ 0.23 9    0   0.0  1.21  1.21 115.113.134.14 5.175.142.66 /server-status?refresh=5
                                                                               HTTP/1.1
                                                                               GET
5-0 5708 0/212/213 _ 0.15 6    0   0.0  0.85  0.85 115.113.134.14 5.175.142.66 /server-status?refresh=5
                                                                               HTTP/1.1
6-0 5709 0/210/210 W 0.16 0    0   0.0  0.84  0.84 127.0.0.1      5.175.142.66 GET /server-status
                                                                               HTTP/1.1
7-0 -    0/0/1     . 0.00 5574 0   0.0  0.00  0.00 127.0.0.1      5.175.142.66 OPTIONS * HTTP/1.0

   --------------------------------------------------------------------------

    Srv  Child Server number - generation
    PID  OS process ID
    Acc  Number of accesses this connection / this child / this slot
     M   Mode of operation
    CPU  CPU usage, number of seconds
    SS   Seconds since beginning of most recent request
    Req  Milliseconds required to process most recent request
   Conn  Kilobytes transferred this connection
   Child Megabytes transferred this child
   Slot  Total megabytes transferred this slot
   --------------------------------------------------------------------------

    Apache/2.2.15 (CentOS) Server at localhost Port 80

Conclusion

Apache’s mod_status module is a very handy monitoring tool for monitoring performance of a web server’s activity and can able to highlight problems itself. For more information read the status page that can be help you to become a more successful web server administrator.

  1. Apache mod_status Homepage

That’s all for mod_status for now, we’ll come up with some more tricks and tips on Apache in future tutorials. Till then stay Geeky and tuned to sfnews.com and don’t forget to add your valuable comments.

GoAccess (A Real-Time Apache and Nginx) Web Server Log Analyzer

GoAccess (A Real-Time Apache and Nginx) Web Server Log Analyzer &-8211; this Article or News was published on this date:2019-05-28 17:35:35 kindly share it with friends if you find it helpful

GoAccess is an interactive and real time web server log analyzer program that quickly analyze and view web server logs. It comes as an open source and runs as a command line in Unix/Linux operating systems. It provides brief and beneficial HTTP (web server) statistics report for Linux administrators on the fly. It also take care of both the Apache and Ngnix web server log formats.

GoAccess parses and analyze the given web server log formats in preferred options including CLF (Common Log Format), W3C format (IIS) and Apache virtual hosts and then generate output of the data to the terminal.

GoAccess Features

It has the following features.

  1. General Statistics, bandwidth etc.
  2. Top Visitors, Visitors Time Distribution, Referring Sites & URLs and 404 or Not Found.
  3. Hosts, Reverse DNS, IP Location.
  4. Operating Systems, Browsers and Spiders.
  5. HTTP Status Codes
  6. Geo Location – Continent/Country/City
  7. Metrics per Virtual Host
  8. Support for HTTP/2 & IPv6
  9. Ability to output JSON and CSV
  10. Incremental log processing and support for large datasets + data persistence
  11. Different Color Schemes

How Do I Install GoAccess?

Presently, the most recent version of GoAccess 0.9.7 is not available from default system package repositories, so to install latest stable version, you need to manually download and compile it from source code under Linux systems as shown:

Install GoAccess from Source

- yum install ncurses-devel glib2-devel geoip-devel
- cd /usr/src
- wget http://tar.goaccess.io/goaccess-0.9.8.tar.gz
- tar zxvf goaccess-0.9.8.tar.gz
- cd goaccess-0.9.8/
- ./configure
- make; make install

Install GoAccess Using Package Manager

The easiest and preferred way to install GoAccess on Linux using the default package manager of your respective Linux distribution.

Note: As I said above, not all distributions will have the most recent version of GoAccess available in the system default repositories..

On RedHat, CentOS and Fedora
- yum install goaccess
- dnf install goaccess    [From Fedora 23+ versions]
On Debian and Ubuntu Systems

GoAccess utility is available since Debian Squeeze 6 and Ubuntu 11.04. To install just run the following command on the terminal.

- apt-get install goaccess

Note: The above command will not always provide you the most latest version. To get the latest stable version of GoAccess, add the official GoAccess Debian & Ubuntu repository as shown:

$ echo "deb http://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/goaccess.list
$ wget -O - http://deb.goaccess.io/gnugpg.key | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install goaccess

How Do I Use GoAccess?

Once, goaccess is installed, execute ‘goaccess‘ command without any arguments will list the help menu.

- goaccess
Sample Output
GoAccess - 0.9.8

Usage: goaccess [ options ... ] -f log_file [-c][-M][-H][-q][-d][...]
The following options can also be supplied to the command:

Log & Date Format Options

  --log-format=        - Specify log format. Inner quotes need to
                                    be escaped, or use single quotes.
  --date-format=      - Specify log date format. e.g.,
                                    %d/%b/%Y
  --time-format=      - Specify log time format. e.g.,
                                    %H:%M:%S

User Interface Options

  -c --config-dialog              - Prompt log/date/time configuration
                                    window.
  -i --hl-header                  - Color highlight active panel.
  -m --with-mouse                 - Enable mouse support on main dashboard.
  --color=fg:bg[attrs, PANEL]>   - Specify custom colors. See manpage for
                                    more details and options.
  --color-scheme=1|2>            - Color schemes: 1 => Grey, 2 => Green.
  --html-report-title=     - Set HTML report page title and header.
  --no-color                      - Disable colored output.
  --no-column-names               - Don't write column names in term
                                    output.
  --no-csv-summary                - Disable summary metrics on the CSV
                                    output.
  --no-progress                   - Disable progress metrics.
  --no-tab-scroll                 - Disable scrolling through panels on TAB.

File Options

  -f --log-file=        - Path to input log file.
  -l --debug-file=      - Send all debug messages to the specified
                                    file.
  -p --config-file=     - Custom configuration file.
  --invalid-requests=   - Log invalid requests to the specified
                                    file.
  --no-global-config              - Don't load global configuration
                                    file.
.....

The easiest way to get the any web server statistics use the flag ‘f‘ with input log file name as shown below. The below command will give you general statistics of your web server logs.

- goaccess -f /var/log/httpd/sfnews.com
- goaccess -f /var/log/nginx/sfnews.com

The above command gives you an complete overview of web server metrics by showing summaries of various reports as panels on a one scrollable view as shown.

Apache Logs Overview

View Web Server Apache LogsView Web Server Apache Logs

View Web Server Apache Logs

Apache Logs by Operating System – Overview

View Web Server Apache LogsView Apache Logs By Operating System

View Apache Logs By Operating System

Apache Logs by Visitor Bandwidth – Overview

View Web Server Apache LogsView Apache Visitor Bandwidth Usage

View Apache Visitor Bandwidth Usage

Apache Logs by Web Browser – Overview

View Web Server Apache LogsView Apache Usage based on Browsers

View Apache Usage based on Browsers

How do I generate Apache HTML report?

To generate an HTML report of your Apache webserver logs, just run it against your web log file.

- goaccess -f /var/log/httpd/access_log > reports.html
View Web Server Apache LogsMonitor Apache Logs Using Web Browser

GoAccess: Monitor Apache Logs Using Web Browser

For more information and usage please visit http://goaccess.io/.

3 Ways to Check Apache Server Status and Uptime in Linux

3 Ways to Check Apache Server Status and Uptime in Linux &-8211; this Article or News was published on this date:2019-05-28 16:44:58 kindly share it with friends if you find it helpful

Apache is a world’s most popular, cross platform HTTP web server that is commonly used in Linux and Unix platforms to deploy and run web applications or websites. Importantly, it’s easy to install and has a simple configuration as well.

Read Also: How to Hide Apache Version Number and Other Sensitive Info

In this article, we will show how to check Apache web server uptime on a Linux system using different methods/commands explained below.

1. Systemctl Utility

Systemctl is a utility for controlling the systemd system and service manager; it is used it to start, restart, stop services and beyond. The systemctl status sub-command, as the name states is used to view the status of a service, you can use it for the above purpose like so:

$ sudo systemctl status apache2	  -Debian/Ubuntu 
- systemctl status httpd	  -RHEL/CentOS/Fedora 
Check Apache Status Using SystemctlCheck Apache Status Using Systemctl

Check Apache Status Using Systemctl

2. Apachectl Utilities

Apachectl is a control interface for Apache HTTP server. This method requires the mod_status (which displays info about the server is performing including its uptime) module installed and enabled (which is the default setting).

On Debian/Ubuntu

The server-status component is enabled by default using the file /etc/apache2/mods-enabled/status.conf.

$ sudo vi /etc/apache2/mods-enabled/status.conf
Check Apache Status Using SystemctlApache Mod_Status Configuration

Apache Mod_Status Configuration

On RHEL/CentOS

To enable server-status component, create a file below.

- vi /etc/httpd/conf.d/server-status.conf

and add the following configuration.

Location "/server-status">
    SetHandler server-status
    -Require  host  localhost		-uncomment to only allow requests from localhost 
/Location>

Save the file and close it. Then restart the web server.

- systemctl restart httpd

If you are primarily using a terminal, then you also need a command line web browser such as lynx or links.

$ sudo apt install lynx		-Debian/Ubuntu
- yum install links		-RHEL/CentOS

Then run the command below to check the Apache service uptime:

$ apachectl status
Check Apache Status Using SystemctlCheck Apache Status Using Apache2ctl

Check Apache Status Using Apache2ctl

Alternatively, use the URL below to view the Apache web server status information from a graphical web browser:

http://localhost/server-status
OR
http:SERVER_IP/server-status

3. ps Utility

ps is a utility which shows information concerning a selection of the active processes running on a Linux system, you can use it with grep command to check Apache service uptime as follows.

Here, the flag:

  • -e – enables selection of every processes on the system.
  • -o – is used to specify output (comm – command, etime – process execution time and user – process owner).
- ps -eo comm,etime,user | grep apache2
- ps -eo comm,etime,user | grep root | grep apache2
OR
- ps -eo comm,etime,user | grep httpd
- ps -eo comm,etime,user | grep root | grep httpd

The sample output below shows that apache2 service has been running for 4 hours, 10 minutes and 28 seconds (only consider the one started by root).

Check Apache Status Using SystemctlCheck Apache Uptime

Check Apache Uptime

Lastly, check out more useful Apache web server guides:

  1. 13 Apache Web Server Security and Hardening Tips
  2. How to Check Which Apache Modules are Enabled/Loaded in Linux
  3. 5 Tips to Boost the Performance of Your Apache Web Server
  4. How to Password Protect Web Directories in Apache Using .htaccess File

In this article, we showed you three different ways to check Apache/HTTPD service uptime on a Linux system. If you have any questions or thoughts to share, do that via the comment section below.

How to Monitor Apache Performance using Netdata on CentOS 7

How to Monitor Apache Performance using Netdata on CentOS 7 &-8211; this Article or News was published on this date:2019-05-28 16:15:57 kindly share it with friends if you find it helpful

Netdata is a free open source, simple yet powerful, and effective real-time system performance monitoring tool for Linux, FreeBSD and MacOS. It supports various plugins for monitoring general server status, applications, web services such as Apache or Nginx HTTP server and so much more.

Read Also: How to Monitor Nginx Performance Using Netdata on CentOS 7

In this article, we will explain how to monitor Apache HTTP server performance using Netdata performance monitoring tool on a CentOS 7 or RHEL 7 distribution. At the end of this article, you will be able to watch visualizations of requests, bandwidth, workers, and other Apache server metrics.

Requirements:

  1. A CentOS 7 Server or RHEL 7 Server with Minimal Install.
  2. Apache HTTP server installation with mod_status module enabled.

Step 1: Install Apache on CentOS 7

1. First start by installing Apache HTTP server from the default software repositories using the YUM package manager.

- yum install httpd

2. After you have installed Apache web server, start it for the first time, check if it is up and running, and enable it to start automatically at system boot using following commands.

- systemctl start httpd
- systemctl enable httpd
- systemctl status httpd

3. If you are running a firewall for example firewalld, you need to open the ports 80 and 443 to allow web traffic to Apache via HTTP and HTTPS respectively, using the commands below.

- firewall-cmd --zone=public --permanent --add-port=80/tcp
- firewall-cmd --zone=public --permanent --add-port=443/tcp
- firewall-cmd --reload 

Step 2: Enable Mod_Status Module in Apache

4. In this step, you need to enable and configure mod_status module in Apache, this is required by Netdata for gathering server status information and statistics.

Open the file /etc/httpd/conf.modules.d/00-base.conf file using your favorite editor.

- vim /etc/httpd/conf.modules.d/00-base.conf

And ensure that the line below is uncommented to enable mod_status module, as shown in the screenshot.

Enable Mod_Status Module in ApacheEnable Mod_Status Module in Apache

Enable Mod_Status Module in Apache

5. Once you’ve enabled mod_status, next you need to create a server-status.conf configuration file for the Apache server status page.

- vim /etc/httpd/conf.d/server-status.conf

Add the following configuration inside the file.

Location "/server-status">
    SetHandler server-status
    -Require host localhost           -uncomment to only allow requests from localhost 
/Location>

Save the file and close. Then restart the Apache HTTPD service.

- systemctl restart httpd

6. Next, you need to verify that the Apache server status and statistics page is working well by using a command-line web browser such as lynx as shown.

- yum install lynx
- lynx http://localhost/server-status   
Enable Mod_Status Module in ApacheCheck Apache Server Status

Check Apache Server Status

Step 3: Install Netdata on CentOS 7

7. Fortunately, there is a kickstarter shell script for painlessly installing netdata from its github repository. This one-liner script downloads a second script which checks your Linux distribution and installs the required system packages for building netdata, then downloads the latest netdata source tree; builds and installs it on your server.

You can start the kickstarter script as shown, the all flag allows for installing required packages for all netdata plugins including the ones for Apache HTTP server.

- bash (curl -Ss https://my-netdata.io/kickstart.sh) all

Note that if your not administering your system as root, you will be prompted to enter your user password for sudo command, and you will also be asked to confirm a number of functions by pressing [Enter].

Enable Mod_Status Module in ApacheInstall Netdata on CentOS 7

Install Netdata on CentOS 7

8. Once the script has completed building and installing netdata, it will automatically start the netdata service via systemd service manager and enables it to start at system boot.

Enable Mod_Status Module in ApacheNetdata Installation Summary

Netdata Installation Summary

By default, netdata listens on port 19999, you will access the web UI using this port. So, open port 19999 in the firewall to access the netdata web UI.

- firewall-cmd --permanent --add-port=19999/tcp
- firewall-cmd --reload 

Step 4: Configure Netdata to Monitor Apache Performance

9. The netdata configuration for Apache plugin is /etc/netdata/python.d/apache.conf, this file is written in YaML format, you can open it using your favorite editor.

- vim /etc/netdata/python.d/apache.conf

The default configuration is just enough to get you started with monitoring your Apache HTTP server.

Enable Mod_Status Module in ApacheNetdata Configuration for Apache

Netdata Configuration for Apache

However, if you have read the documentation, and made any changes to it, restart the netdata service to effect the changes.

- systemctl restart netdata 

Step 5: Monitor Apache Performance Using Netdata

10. Next, open a web browser and use the following URL to access the netdata web UI.

http://domain_name:19999
OR
http://SERVER_IP:19999

From the netdata dashboard, search for “Apache local” on the right hand side list of plugins, and click on it to start monitoring your Apache server. You will be able to watch visualizations of requests, bandwidth, workers, and other server statistics, as shown in the following screenshot.

Enable Mod_Status Module in ApacheMonitor Apache Performance Using Netdata

Monitor Apache Performance Using Netdata

Netdata Github repository: https://github.com/firehol/netdata

That’s all! In this article, we’ve explained how to monitor Apache performance using Netdata on CentOS 7. If you have any questions or additional thoughts to share, please reach us via the comment form below.