Use logwatch to keep an eye on your logfiles

Discussion in 'Tutorials and Guides' started by wlanboy, Aug 21, 2013.

  1. wlanboy

    wlanboy Content Contributer

    2,126
    1,169
    May 16, 2013
    It is one of the tasks I want to do but never try to finish: Check logfiles.

    Look through all the logfiles to see if something happend that should not happen.

    It is a job which is boring and so a single important line can splip through your scrolling.

    But linux-like: There is a tool for it - called logwatch.

    1. Installation

      sudo apt-get install logwatch libdate-manip-perl (yum: perl-DateManip)
      sudo mkdir /var/cache/logwatch
      sudo cp /usr/share/logwatch/default.conf/logwatch.conf /etc/logwatch/conf/

      Thats all. The cache dir and the default config are not part of the installer because logwatch can be easily used by command line. You do need them only if you want to run a cronjob.


      perl-DateManip is a easy option to define date ranges.
    2. Configuration
      Code:
      sudo nano /etc/logwatch/conf/logwatch.conf
      
      Things you should alter:


      Output = mail
      Format = html
      MailTo = [email protected]
      MailFrom = [email protected]
      Encode = base64

      #Range = yesterday
      Range = between -7 days and -1 days
      Detail = High
      Service = All

      Service can be all daemons like sshd.
       

    3. Cron


      00 20 * * * /usr/sbin/logwatch --mailto [email protected]

    A simple call for the command line would be:
     


    logwatch --service sshd --range today --detail 10

    output is something like:


    ################### Logwatch 7.4.0 (03/01/11) ####################
    Processing Initiated: Wed Aug 21 07:19:18 2013
    Date Range Processed: today
    ( 2013-Aug-21 )
    Period is day.
    Detail Level of Output: 10
    Type of Output/Format: stdout / text
    Logfiles for Host: servnl
    ##################################################################

    --------------------- SSHD Begin ------------------------

    SSHD Killed: 2 Time(s)

    Users logging in through sshd:
    aname:
    8.8.8.8 (a-host-name): 5 times
    bname:
    9.9.9.9 (another-host-name): 2 times
    ---------------------- SSHD End -------------------------


    ###################### Logwatch End #########################


    Or postfix:


    logwatch --service postfix --range today --detail 10

    Code:
     --------------------- Postfix Begin ------------------------
    
     ****** Summary *************************************************************************************
    
        7.760K  Bytes accepted                               7,946
        7.760K  Bytes delivered                              7,946
     ========   ==================================================
    
           20   Accepted                                   100.00%
     --------   --------------------------------------------------
           20   Total                                      100.00%
     ========   ==================================================
    
           20   Removed from queue
           20   Delivered
    
     ****** Detail (1) **********************************************************************************
    
           20   Delivered -------------------------------------------------------------------------------
           20      wlanboy.com
    
     === Delivery Delays Percentiles ============================================================
                         0%       25%       50%       75%       90%       95%       98%      100%
     --------------------------------------------------------------------------------------------
     Before qmgr       0.00      0.00      4.00      9.50     10.10     11.00     11.00     11.00
     In qmgr           0.00      0.00      0.01      0.03      0.04      0.04      0.04      0.04
     Conn setup        0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
     Transmission      0.01      0.01      0.01      0.02      0.02      0.03      0.03      0.03
     Total             0.01      0.02      4.07      9.53     10.10     11.00     11.00     11.00
     ============================================================================================
     ---------------------- Postfix End -------------------------
    
    
    A great tool to get weekly reports about all services which are running on a vps.
     
    Last edited by a moderator: Aug 21, 2013
    peterw, Abdussamad, MannDude and 6 others like this.
  2. clarity

    clarity Active Member

    351
    125
    May 15, 2013
    This:


    sudo nano /etc/logwatch/logwatch.conf
    Should be:

    Code:
    sudo nano /etc/logwatch/conf/logwatch.conf
     
    wlanboy likes this.
  3. imperio

    imperio New Member

    70
    19
    May 15, 2013
    Is it configurable like if specific pattern is found on the log file alert via mail ?
     
  4. wlanboy

    wlanboy Content Contributer

    2,126
    1,169
    May 16, 2013
    Fixed it. Thank you for pointing to the missing subfolder.

    You can customize logwatch: http://www.stellarcore.net/logwatch/tabs/docs/HOWTO-Customize-LogWatch.html

    E.g.:

    I want to watch a new service:

    So I create a new file in:


    /etc/logwatch/conf/services/

    Content would be like:


    Title = "mynewservice"
    LogFile = messages
    *OnlyService = serviceD #filter everything that is not logged via this name
    *RemoveHeaders # filter time stamp, hostname, ...

    Line two is a head cracker...

    You have to set the log file group that is used, which tells logwatch which file it should look in the /etc/logwatch/logfiles/.

    Conent of the messages.conf is:


    # What actual file? Defaults to LogPath if not absolute path....
    LogFile = messages

    # If the archives are searched, here is one or more line
    # (optionally containing wildcards) that tell where they are...
    #If you use a "-" in naming add that as well -mgt
    Archive = messages.*
    Archive = archiv/messages.*
    Archive = messages-*
    Archive = archiv/messages-*

    # Expand the repeats (actually just removes them now)
    *ExpandRepeats

    # Now, lets remove the services we don't care about at all...
    # Comma separated list works best -mgt
    *RemoveService = talkd,telnetd,inetd,nfsd,/sbin/mingetty,netscreen,NetScreen

    # Keep only the lines in the proper date range...
    *ApplyStdDate

    # vi: shiftwidth=3 tabstop=3 et


    So the "messages" do include all "/var/log/messages.*" files.

    Now we need a perl script saved in .../scripts/services/mynewservice.pl

    Content would be something like: (pam for example)


    #!/usr/bin/perl

    ##########################################################################
    # $Id: pam,v 1.11 2008/03/24 23:31:26 kirk Exp $
    ##########################################################################

    #####################################################
    ## Copyright (c) 2008 Kirk Bauer
    ## Covered under the included MIT/X-Consortium License:
    ## http://www.opensource.org/licenses/mit-license.php
    ## All modifications and contributions by other persons to
    ## this script are assumed to have been donated to the
    ## Logwatch project and thus assume the above copyright
    ## and licensing terms. If you want to make contributions
    ## under your own copyright or a different license this
    ## must be explicitly stated in the contribution an the
    ## Logwatch project reserves the right to not accept such
    ## contributions. If you have made significant
    ## contributions to this script and want to claim
    ## copyright please contact [email protected]
    #########################################################

    $Detail = $ENV{'LOGWATCH_DETAIL_LEVEL'} || 0;

    while (defined($ThisLine = <STDIN>)) {
    if ( ( $ThisLine =~ /^pam_get_user: no username obtained$/ ) or
    ( $ThisLine =~ /^pam_end: NULL pam handle passed/ ) ) {
    # We don't care about these
    }
    elsif ( $ThisLine =~ s/^FAILED LOGIN SESSION FROM ([^ ]+) FOR .*$/$1/ ) {
    $FailedLogins{$ThisLine}++;
    }
    else {
    # Report any unmatched entries...
    push @OtherList,$ThisLine;
    }
    }

    if ( (keys %FailedLogins) and ($Detail >= 10) ) {
    print "\nFailed Login Sessions:\n";
    foreach $ThisOne (keys %FailedLogins) {
    print " " . $FailedLogins{$ThisOne} . " from " . $ThisOne;
    }
    }

    if ($#OtherList >= 0) {
    print "\n**Unmatched Entries**\n";
    print @OtherList;
    }

    exit(0);

    # vi: shiftwidth=3 tabstop=3 syntax=perl et
    # Local Variables:
    # mode: perl
    # perl-indent-level: 3
    # indent-tabs-mode: nil
    # End:

    There are a lot of examples in /usr/share/logwatch/scripts/services.

    I like to work with if statements to search for events:


    if (
    ( $TheLine =~ m/new connection/ )
    ) {
    print "a new connectiont!"
    }

    The scripts are quite simple. You get one STDIN and you have to print  to STDOUT. Just print every information you want to.
     
    Last edited by a moderator: Aug 22, 2013
    scv likes this.
  5. drmike

    drmike 100% Tier-1 Gogent

    8,573
    2,717
    May 13, 2013
    You are on a roll @wlanboy.  
     
    perennate likes this.
  6. scv

    scv Massive Nerd Verified Provider

    205
    98
    May 30, 2013
    scv
    The default logwatch on FreeBSD is great. If only more Linux distributions offered this sort of functionality out of the box.
     
  7. Maximum_VPS

    Maximum_VPS New Member Verified Provider

    48
    4
    Jun 18, 2013
    @wlanboy another excellent Guide :)
     
  8. wlanboy

    wlanboy Content Contributer

    2,126
    1,169
    May 16, 2013
    Just write about things that easy my work.

    Thank you.

    I start this series of short guides to draw our attention to this cool little helpers.
     
  9. johng

    johng New Member

    7
    4
    Aug 15, 2013
    **Noob alert**


    What is the default logwatch on FreeBSD? Is it logwatch or a BSD equivalent?


    Thanks.


    **Noob alert over. You may now return to your regularly scheduled programming.**
     
  10. scv

    scv Massive Nerd Verified Provider

    205
    98
    May 30, 2013
    scv
    They aren't the same service but they offer similar functionality. By default it gives you a security log and a general log. Here's an example from a desktop machine I use at my day job:

    Daily run

    Security check:

    Since the machine's mostly idle there isn't anything very exciting or relevant in there but I am sure you can get the general gist of its purpose.
     
    johng likes this.
  11. CraigA

    CraigA New Member Verified Provider

    28
    5
    Aug 22, 2013
    Thanks for this, I'm going to check it out.
     
  12. VPSCorey

    VPSCorey New Member Verified Provider

    271
    57
    Jul 10, 2013
    I'll evangelize Splunk.com for playing with logs as well.  There's a free edition for 500MB a day of logs.
     
  13. trexos

    trexos New Member

    23
    1
    May 16, 2013
    *noob alert*

    I get this error:


    Can not open HTML Header at /usr/share/logwatch/default.conf/html/header.html: No such file or directory

    I read that this is a debian error. How can I fix it?

    Edit #1: I fixed it with downloading and copying the html files from the newest release.

    But know I have this error:


    [email protected]:~# logwatch --service sshd --range today --detail 10
    [email protected]:~# logwatch --service sshd --range today --detail 10

    I don't get a result.
     
    Last edited by a moderator: Sep 3, 2013
  14. wlanboy

    wlanboy Content Contributer

    2,126
    1,169
    May 16, 2013
    If you did all the steps the default output of logwatch is "send via email".

    Do you have a running smtp server?

    Run following command to see if mail delivery is working:


    sudo less /var/log/mail.log

    Don't want to install a whole mailserver?

    Install nullmailer and use an existing mail account for that. Look at this tutorial.

    If you want to see the output (for testeing add following parameter --output):


    logwatch --service sshd --range today --detail 10 --output stdout

    It will forward the output to the console.
     
  15. tonyg

    tonyg New Member

    379
    134
    Aug 28, 2013
    Any way to correct the misspelling on the title?

    logfiles not logiles.
     
  16. wlanboy

    wlanboy Content Contributer

    2,126
    1,169
    May 16, 2013
    I asked the admin ... cannot edit anything.
     
  17. peterw

    peterw New Member

    800
    189
    Jun 14, 2013
    Missed this great tutorial. Thank you.
     
  18. wlanboy

    wlanboy Content Contributer

    2,126
    1,169
    May 16, 2013
    And there is an open source one doing the same: Fluentd.