How to we can access the (centos) server error log without logging in server?

Hello,

In order to do that you need to setup your own Centralized Logging system.

I would recommend taking a look at the following step by step tutorials and picking up the solution that would best match your needs:

https://www.digitalocean.com/community/tutorials/building-for-production-web-applications-centralized-logging

https://www.digitalocean.com/community/tutorials/how-to-centralize-logs-with-rsyslog-logstash-and-elasticsearch-on-ubuntu-14-04

https://www.digitalocean.com/community/tutorial_series/centralized-logging-with-elk-stack-elasticsearch-logstash-and-kibana-on-ubuntu-14-04

Regards, Bobby

To store Apache2 server logs in a database for easy access via an admin dashboard, you will need to follow a series of steps. This includes configuring Apache to write logs in a way that can be efficiently parsed and inserted into a database, and setting up a script to do so. Below, I’ll guide you through a high-level process using MySQL as an example database:

Step 1: Choose a Logging Format

Apache logs can be customized using various formats. To make it easier to parse and store in a database, you might want to standardize the format. You can define a custom log format in Apache’s configuration file.

1. Open Apache Configuration:

sudo nano /etc/apache2/apache2.conf

2. Define a Custom Log Format: Add a line like the following to define a log format that simplifies parsing:

LogFormat "%h %l %u %t \"%r\" %>s %b" combined_custom

3. Configure Apache to Use This Format: In the same or relevant virtual host file, specify the log format:

CustomLog ${APACHE_LOG_DIR}/access.log combined_custom

4. Restart Apache to apply changes:

sudo systemctl restart apache2

Step 2: Set Up a Database Schema

Create a table in your MySQL database to store the log data.

1. Log in to MySQL:

mysql -u root -p

2. Create a table:

CREATE TABLE apache_logs (
  id INT AUTO_INCREMENT PRIMARY KEY,
  host VARCHAR(255),
  logname VARCHAR(255),
  user VARCHAR(255),
  time TIMESTAMP,
  request VARCHAR(255),
  status INT,
  size INT
);

Step 3: Create a Script to Parse Logs and Insert Into Database

Write a script that reads the Apache log file, parses each line, and inserts the data into the database. Here’s an example in Python:

import pymysql
import re

# Database connection
conn = pymysql.connect(host='localhost', user='user', password='password', db='database')
cursor = conn.cursor()

# Open the Apache log file
with open('/var/log/apache2/access.log', 'r') as file:
    for line in file:
        # Parse the line using regex or split
        parts = re.split(r'\s+', line)
        host = parts[0]
        logname = parts[1]
        user = parts[2]
        time = parts[3] + ' ' + parts[4]
        request = parts[5] + ' ' + parts[6] + ' ' + parts[7]
        status = int(parts[8])
        size = int(parts[9])

        # Insert into database
        sql = "INSERT INTO apache_logs (host, logname, user, time, request, status, size) VALUES (%s, %s, %s, STR_TO_DATE(%s, '[%d/%b/%Y:%H:%i:%s %z]'), %s, %s, %s)"
        cursor.execute(sql, (host, logname, user, time, request, status, size))

# Commit and close
conn.commit()
cursor.close()
conn.close()

Step 4: Schedule the Script

Use cron on Linux to schedule this script to run at regular intervals (e.g., every hour).

crontab -e

0 * * * * /usr/bin/python3 /path/to/your/script.py

Step 5: Access Logs in Your Admin Dashboard

Now that the logs are stored in your database, you can easily query them and display them in your admin dashboard using whatever backend technology your dashboard is built with (e.g., PHP, Python, etc.).

This process automates the collection of Apache logs into a database, simplifying access via an administrative interface and allowing for more complex queries and analysis than standard file-based logging.