django-chunk-upload-handlers 0.0.6

Chunking Django file handlers for S3 and ClamAV service uploads

This library provides two Django file upload handlers.

The first of these, s3 provides chunk uploading to an AWS S3 bucket and is heavily based on https://pypi.org/project/s3chunkuploader/

The second provides chunk uploading to a ClamAV rest endpoint provided by - https://github.com/uktrade/dit-clamav-rest/ although it could be used with other compatibile projects.

If used together, the results of the anti virus check are written to the object uploaded to S3.

Installation

pip install django-chunk-upload-handlers

Usage

Using in a Django logging configuration:

INSTALLED_APPS = [
    ...
    "django_chunk_upload_handlers",
]

FILE_UPLOAD_HANDLERS = (
    "django_chunk_upload_handlers.clam_av.ClamAVFileUploadHandler",
    "django_chunk_upload_handlers.s3.S3FileUploadHandler",
)  # Order is important

Dependencies

This project is a Django app and depends on the Django package.

The s3 file handler depends on boto3 and django-storages

settings.DEFAULT_FILE_STORAGE must be set to "storages.backends.s3boto3.S3Boto3Storage" or a class that derives from it.

Settings

S3

AWS_ACCESS_KEY_ID CHUNK_UPLOADER_AWS_ACCESS_KEY_ID

Provide either for the AWS access key required. CHUNK_UPLOADER_AWS_ACCESS_KEY_ID is preferred if both are set.

AWS_SECRET_ACCESS_KEY CHUNK_UPLOADER_AWS_SECRET_ACCESS_KEY

Provide either for the AWS access secret key required. CHUNK_UPLOADER_AWS_SECRET_ACCESS_KEY is preferred if both are set.

AWS_STORAGE_BUCKET_NAME CHUNK_UPLOADER_AWS_STORAGE_BUCKET_NAME

The S3 bucket to use for uploads. CHUNK_UPLOADER_AWS_STORAGE_BUCKET_NAME is preferred if both are set.

AWS_REGION CHUNK_UPLOADER_AWS_REGION

The AWS region to use. CHUNK_UPLOADER_AWS_REGION is preferred if both are set.

S3_ROOT_DIRECTORY CHUNK_UPLOADER_S3_ROOT_DIRECTORY

The directory path to use as root for uploads. CHUNK_UPLOADER_S3_ROOT_DIRECTORY is preferred if both are set.

ADD_TIMESTAMP_TO_OBJECT_NAME Adds a timestamp to uploaded object’s file name if set. Defaults to True.

CHUNK_UPLOADER_RAISE_EXCEPTION_ON_VIRUS_FOUND Defines whether or not to throw an exception if a virus is found. Defaults to False.

ClamAV

CLAM_AV_USERNAME The ClamAV service username.

CLAM_AV_PASSWORD The ClamAV service password.

CLAM_AV_DOMAIN The domain to use for the ClamAV service. Note, this is domain only so test.com rather than https://test.com

CLAM_PATH The path to the ClamAV service (used with the domain defined in the setting above). Defaults to /v2/scan-chunked

CLAM_AV_IGNORE_EXTENSIONS A list of file extensions to not process with ClamAV. Defaults to an empty list.

CLAM_USE_HTTP Use http rather than https. Should not be used in production environments. Defaults to False.

Usage with file fields

The package provides a validator for use with form and model fields.

The CHUNK_UPLOADER_RAISE_EXCEPTION_ON_VIRUS_FOUND should not be set to True when using this validator.

from django import forms
from django_chunk_upload_handlers.clam_av import validate_virus_check_result


class ExampleForm(forms.Form):
    example_form_field = forms.FileField(
        validators=[validate_virus_check_result, ]
    )

from django.db import models

class ExampleModel(models.Model):
    example_model_field = models.FileField(
        max_length=10,
        validators=[validate_virus_check_result, ],
    )

The validation message will display ‘A virus was found’ if a virus is detected. This message is a translation string.

Tests

$ pip install -r requirements.txt
$ tox