accompanist 1.2.0

Analysis AWS WAF logs and generate a report

Introduction

---

What is Accomapnist?

Accompanist is an AWS WAF log analysis tool.
It enables you to analyze AWS WAF logs and automatically generate comprehensive analysis reports in just three simple steps. If you're using AWS WAF and CloudWatch Logs, this tool can be helpful for instant log analysis and help you understand the trends of blocked or counted accesses.

• Reporting Items
  • Histgram of requests
    • Total number
    • Number by each WAF rule provider
  • Top 5 for the following categories:
    • Rule group
    • Rule
    • Paths name
    • User agent
    • IP addressws
    • Country codes
  • Hit count on specific paths
  • Comment
    • An optional field where you can write additional notes

Prerequisite

• Conditions

  • WAF: AWS WAFv2 and Third Party WAF (using WebACL)
  • Logging: CloudWatch Logs or S3
  • Support Action: BLOCK, COUNT or EXCLUDED_AS_COUNT

• Requirement of Client Environment

  • IAM Role/User: including permissions to execute as follows
    • (1) start_query of Logs Insights
    • (2) get_query_result of Logs Insights
    • The example policy is noted the last

Get Started

---

Install

$ pip install accompanist

Usage

1. First, configure the settings using the following command; this will generate a config.json file.

$ accompanist init

• You must set the log group name using this command, or you can choose to edit the config.json file directly.

2. Second, retrieve the query results with the following commnad.

• For example, to retrieve the BLOCK log for the past three days

$ accompanist listen -a BLOCK -d 3

3. Finally, generate a report in PDF format.

$ accompanist play

Uninstall

$ pip uninstall accompanist

Precaution

• This tool works with AWS WAFv2 and does not support v1.
• For AWS Managed Rules, you need to append "AWS" to the prefix, or the histogram works wrong.
• Using the "COUNT" action, the results in double counting.
• The logs may not be correctly calculated if you use the Label function in WAF.

Subcommnads

---

init

• Configure a CWL log group setting and more.

Usage:

accompanist init  # interactive inputting process

Options:

  -l, --log-group TEXT  Set a CloudWatch Logs Log group name.  [required]
  -p, --path TEXT       Set a URI path for counts that is blocked/counted.[required]
  -c, --comment TEXT    Set a comment for report.  [required]
  -h, --help            Show this message and exit.

listen

• Get a WAF log file in CSV format.

Usage:

accompanist listen [OPTIONS]

Options:

  -a, --action [BLOCK|COUNT]  Chose an action type of AWS WAF. The default is
                              "BLOCK".  [required]
  -d, --days INTEGER          Set a number of the past days until today for
                              analysis target period.
  -s, --start_time INTEGER    Set a UNIX time of the oldest time for analysis
                              target period (instead of "--days").
  -e, --end_time INTEGER      Set a UNIX time of the latest time for analysis
                              target period (instead of "--days").
  -r, --raw-log               Output raw log file and the excerpted log file
                              for debugging.
  -h, --help                  Show this message and exit.

hear

Note: This is experimental feature.

• Get a WAF log file in CSV format from S3.

Usage:

accompanist hear [OPTIONS]

Options:

  -a, --action [BLOCK|COUNT]  Chose an action type of AWS WAF. The default is
                              "BLOCK".  [required]
  -d, --days INTEGER          Set a number of the past days until today for
                              analysis target period.
  -s, --start_time INTEGER    Set a UNIX time of the oldest time for analysis
                              target period (instead of "--days").
  -e, --end_time INTEGER      Set a UNIX time of the latest time for analysis
                              target period (instead of "--days").
  -h, --help                  Show this message and exit.

play

• Analysis WAF logs and generate a report.

Usage:

accompanist play [OPTIONS]

Options:

  -c, --colorful                  Set a random color of report theme (instead
                                  of color).
  -d, --color TEXT                Customize a color of report theme with color
                                  code,  (e.g.) #cccccc.
  -m, --mask-ip                   Mask IP addresses on pie chart.
  -u, --utc-offset INTEGER        Set a number of UTC offest. The defaut
                                  offset is UTC+9 (Asia/Tokyo).
  -y, --y-limit [50|100|500|1000]
                                  Adjust a Y-axis max limitation for
                                  histograms due to many requests.
  -h, --help                      Show this message and exit.

Appendix

---

IAM Policy for CWL

Here is an example of IAM policy with minimum permissions.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "logs:StartQuery",
            "Resource": [
                "arn:aws:logs:<region>:<aws-acount>:log-group:<log-group-name>:*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "logs:GetQueryResults",
            "Resource": "*"
        }
    ]
}

Note: The <region>, <aws-acount> and <log-group-name> in this sample policy should be replaced.

IAM Policy for S3

Here is an example of IAM policy with minimum permissions.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<waf-log-bucket>"
        },
        {
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::<waf-log-bucket>/*"
        }
    ]
}

Note: The <waf-log-bucket>in this sample policy should be replaced.

License

The MIT License

Copyright 2023 Itsuki Yutaka

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.