espebra/filebin2: Filebin is a web application that facilitates convenient file sharing over the web.

Filebin2 is a web application that facilitates convenient file sharing over the web. It is the software that powers https://filebin.net/. It is still in development status and will see breaking changes.

Why filebin2?
Getting started

* With Docker Compose * With the container image * From the binary

Development environment
Usage

* Configuration

Integrations

Why filebin2?

A couple of (in hindsight) bad architectural decisions in the previous version of filebin paved the road for filebin2. Filebin2 is using a PostgreSQL database to handle meta data and S3 to store files. I decided to move to a new repository because of breaking changes from the previous verson of filebin.

Getting started

Filebin2 requires read write access to an S3 bucket for file storage and a PostgreSQL database that it will use for meta data. The Filebin application itself is written in Go and builds to a single binary that is configured using command line arguments and/or environment variables.

Using Docker Compose

The quickest way to get a fully working setup is to use the included docker-compose.yml, which builds Filebin from source and starts it together with PostgreSQL and Stupid Simple S3):

docker compose up --build

This makes Filebin available at http://localhost:8080/ with an admin interface at http://admin:changeme@localhost:8080/admin.

Review and adjust the environment variables in docker-compose.yml to suit your needs. See the Configuration section for all available options.

Using the container image

Multi-arch container images (linux/amd64 and linux/arm64) are published to GitHub Container Registry for each release. The container images are distroless and contain only the filebin binary:

ghcr.io/espebra/filebin2:latest

Run the container with the required environment variables pointing to your PostgreSQL database and S3 bucket:

docker run -p 8080:8080 \
  -e FILEBIN_DATABASE_HOST=db.example.com \
  -e FILEBIN_DATABASE_NAME=filebin \
  -e FILEBIN_DATABASE_USERNAME=filebin \
  -e FILEBIN_DATABASE_PASSWORD=secret \
  -e FILEBIN_S3_ENDPOINT=s3.example.com \
  -e FILEBIN_S3_REGION=us-east-1 \
  -e FILEBIN_S3_BUCKET=filebin \
  -e FILEBIN_S3_ACCESS_KEY=accesskey \
  -e FILEBIN_S3_SECRET_KEY=secretkey \
  -e FILEBIN_LISTEN_HOST=0.0.0.0 \
  -e [email protected] \
  ghcr.io/espebra/filebin2:latest

Using the binary

Pre-built binaries for Linux and macOS (amd64 and arm64) are attached to each release. Download the appropriate binary for your platform:

| Binary | Platform | | ------ | -------- | | filebin2-linux-amd64 | Linux x86_64 | | filebin2-linux-arm64 | Linux ARM64 | | filebin2-darwin-amd64 | macOS x86_64 | | filebin2-darwin-arm64 | macOS ARM64 (Apple Silicon) |

Make the binary executable and run it with the required flags:

chmod +x filebin2-linux-amd64

./filebin2-linux-amd64 \ --db-host db.example.com \ --db-name filebin \ --db-username filebin \ --db-password secret \ --s3-endpoint s3.example.com \ --s3-region us-east-1 \ --s3-bucket filebin \ --s3-access-key accesskey \ --s3-secret-key secretkey \ --listen-host 0.0.0.0 \ --contact [email protected]

Configuration

Filebin can be configured using command line arguments or environment variables. Environment variables use the FILEBIN_ prefix with uppercase letters and underscores instead of hyphens. Command line flags take precedence over environment variables.

General

Base URL

Environment Variable: FILEBIN_BASEURL
Command Line Argument: --baseurl
Default: https://filebin.net

The base URL to use, which impacts URLs that are presented to the user for files and bins. It needs to point to the hostname of the filebin instance.

Contact

Environment Variable: FILEBIN_CONTACT
Command Line Argument: --contact
Default: (required)

The e-mail address to show on the website on the contact page.

Expiration

Environment Variable: FILEBIN_EXPIRATION
Command Line Argument: --expiration
Default: 604800

Bin expiration time in seconds since the last bin update. Bins will be inaccessible after this time, and files will be removed by the lurker.

Temporary Directory

Environment Variable: FILEBIN_TMPDIR
Command Line Argument: --tmpdir
Default: /tmp

Directory for temporary files for upload and download.

Temporary Directory Capacity Threshold

Environment Variable: FILEBIN_TMPDIR_CAPACITY_THRESHOLD
Command Line Argument: --tmpdir-capacity-threshold
Default: 4.0

Workspace capacity threshold multiplier. A workspace must have at least this multiplier times the file size available to be selected (e.g., 4.0 requires 4x the file size available).

Manual Approval

Environment Variable: FILEBIN_MANUAL_APPROVAL
Command Line Argument: --manual-approval
Default: false

If enabled, the administrator needs to manually approve new bins before files and archives can be downloaded. Bin and file operations except downloading are accepted while a bin is pending approval. This is a mechanism added to limit abuse. The API request used to approve a bin is an authenticated PUT /admin/approve/{bin}.

Require Verification Cookie

Environment Variable: FILEBIN_REQUIRE_VERIFICATION_COOKIE
Command Line Argument: --require-verification-cookie
Default: false

If enabled, a warning page will be shown before a user can download files. If the user accepts to continue past the warning page, a cookie will be set to identify that the user understands the risk of downloading files.

Verification Cookie Lifetime

Environment Variable: FILEBIN_VERIFICATION_COOKIE_LIFETIME
Command Line Argument: --verification-cookie-lifetime
Default: 365

Number of days before cookie expiration. See --require-verification-cookie.

Expected Cookie Value

Environment Variable: FILEBIN_EXPECTED_COOKIE_VALUE
Command Line Argument: --expected-cookie-value
Default: 2024-05-24

Which cookie value to expect to avoid showing a warning message. See --require-verification-cookie.

GeoIP City Database

Environment Variable: FILEBIN_MMDB_CITY
Command Line Argument: --mmdb-city
Default: (not set)

The path to an mmdb formatted geoip database like GeoLite2-City.mmdb. This is optional.

GeoIP ASN Database

Environment Variable: FILEBIN_MMDB_ASN
Command Line Argument: --mmdb-asn
Default: (not set)

The path to an mmdb formatted geoip database like GeoLite2-ASN.mmdb. This is optional.

Allow Robots

Environment Variable: FILEBIN_ALLOW_ROBOTS
Command Line Argument: --allow-robots
Default: false

If enabled, the X-Robots-Tag response header will allow search engines to index and show Filebin in search results. Otherwise, robots will be instructed to not show files and bins in search results.

Pre-Upload Hook

Environment Variable: FILEBIN_PRE_UPLOAD_HOOK
Command Line Argument: --pre-upload-hook
Default: (not set)

Command to execute before every file upload is accepted. The command runs after the file has been received from the client and its content type and checksums have been calculated, but before it is stored in S3 or its metadata persisted to the database. It is intended for validation, for example checking the checksums against a blocklist or a malware scanning service. It is invoked with the following named arguments: --bin-id, --filename, --content-type, --size, --md5, --sha1, and --sha256. The checksums are hex encoded.

The exit code determines the outcome:

1 — Reject the upload with HTTP 403 (Forbidden). The last line of stdout from the command is returned to the client as the response message.
0 — Accept the upload.
Any other exit code — Accept the upload. The exit code and output are logged.

Only exit code 1 rejects an upload. A command that cannot be started, or that does not finish within the timeout, is logged and the upload is accepted, so that a broken or hung hook cannot block uploads. An example hook script is provided in misc/pre-upload-hook-example.

Pre-Upload Hook Timeout

Environment Variable: FILEBIN_PRE_UPLOAD_HOOK_TIMEOUT
Command Line Argument: --pre-upload-hook-timeout
Default: 5s

Timeout for the pre-upload hook command execution. If the command does not complete within this duration, it is killed and the timeout is logged, but the upload is accepted so that a slow or hung hook does not block uploads. The value is specified using Go duration format, examples: 5s, 30s, 1m.

Post-Upload Hook

Environment Variable: FILEBIN_POST_UPLOAD_HOOK
Command Line Argument: --post-upload-hook
Default: (not set)

Command to execute after every successful file upload. The command runs after the file has been stored in S3 and its metadata persisted to the database, and is intended for notifications and post-processing (for example, triggering an indexing job or webhook). It is invoked with the following named arguments: --bin-id, --filename, --content-type, --size, --md5, --sha1, and --sha256. The checksums are hex encoded.

The hook does not affect the response to the client: any non-zero exit code or stdout/stderr is logged but the upload is always reported as successful. An example hook script is provided in misc/post-upload-hook-example.

Post-Upload Hook Timeout

Environment Variable: FILEBIN_POST_UPLOAD_HOOK_TIMEOUT
Command Line Argument: --post-upload-hook-timeout
Default: 10s

Timeout for the post-upload hook command execution. If the command does not complete within this duration, it is killed and the failure is logged, but the upload is still reported as successful to the client. The value is specified using Go duration format, examples: 5s, 30s, 1m.

Limits

Limit File Downloads

Environment Variable: FILEBIN_LIMIT_FILE_DOWNLOADS
Command Line Argument: --limit-file-downloads
Default: 0

Limit the number of downloads per file. 0 means no limit. If the value is 100, then each file can be downloaded 100 times before further downloads are rejected.

Limit Storage

Environment Variable: FILEBIN_LIMIT_STORAGE
Command Line Argument: --limit-storage
Default: 0

Limit the storage capacity that filebin will use. 0 means no limit. If set to "200GB", filebin will allow file uploads until the total storage surpasses 200 GB. New file uploads will be rejected until storage consumption is below 200 GB again.

Reject File Extensions

Environment Variable: FILEBIN_REJECT_FILE_EXTENSIONS
Command Line Argument: --reject-file-extensions
Default: (not set)

A whitespace separated list of file extensions that will be rejected. Example: "exe bat dll".

HTTP Server

Listen Host

Environment Variable: FILEBIN_LISTEN_HOST
Command Line Argument: --listen-host
Default: 127.0.0.1

Which IP address Filebin will bind to. The default value of 127.0.0.1 is a safe default that does not expose Filebin outside of the host.

Listen Port

Environment Variable: FILEBIN_LISTEN_PORT
Command Line Argument: --listen-port
Default: 8080

Which port Filebin will listen to. The default of 8080 does not require privileged access.

Access Log

Environment Variable: FILEBIN_ACCESS_LOG
Command Line Argument: --access-log
Default: /var/log/filebin/access.log

Path to a filename for the access log output.

Proxy Headers

Environment Variable: FILEBIN_PROXY_HEADERS
Command Line Argument: --proxy-headers
Default: false

If enabled, the client IP will be read from the proxy headers provided in the incoming HTTP requests. This should only be enabled if there is an HTTP proxy running in front of Filebin that uses proxy headers to tell Filebin the original client IP address.

Timeouts

Read Timeout

Environment Variable: FILEBIN_READ_TIMEOUT
Command Line Argument: --read-timeout
Default: 1h

Read timeout for the HTTP server. File uploads need to complete within this timeout before they are terminated.

Read Header Timeout

Environment Variable: FILEBIN_READ_HEADER_TIMEOUT
Command Line Argument: --read-header-timeout
Default: 2s

Read header timeout for the HTTP server.

Write Timeout

Environment Variable: FILEBIN_WRITE_TIMEOUT
Command Line Argument: --write-timeout
Default: 1h

Write timeout for the HTTP server.

Idle Timeout

Environment Variable: FILEBIN_IDLE_TIMEOUT
Command Line Argument: --idle-timeout
Default: 30s

Idle timeout for the HTTP server.

Database

Database Host

Environment Variable: FILEBIN_DATABASE_HOST
Command Line Argument: --db-host
Default: (required)

Which PostgreSQL host to connect to. This can be an IP address or a hostname.

Database Port

Environment Variable: FILEBIN_DATABASE_PORT
Command Line Argument: --db-port
Default: 5432

The port to use when connecting to the PostgreSQL database.

Database Name

Environment Variable: FILEBIN_DATABASE_NAME
Command Line Argument: --db-name
Default: (required)

The name of the PostgreSQL database to use.

Database Username

Environment Variable: FILEBIN_DATABASE_USERNAME
Command Line Argument: --db-username
Default: (required)

The username to use when authenticating to the PostgreSQL database.

Database Password

Environment Variable: FILEBIN_DATABASE_PASSWORD
Command Line Argument: --db-password
Default: (required)

The password to use when authenticating to the PostgreSQL database.

Max Open Connections

Environment Variable: FILEBIN_DATABASE_MAX_OPEN_CONNS
Command Line Argument: --db-max-open-conns
Default: 25

Maximum number of open database connections.

Max Idle Connections

Environment Variable: FILEBIN_DATABASE_MAX_IDLE_CONNS
Command Line Argument: --db-max-idle-conns
Default: 25

Maximum number of idle database connections.

Connection Max Lifetime

Environment Variable: FILEBIN_DATABASE_CONN_MAX_LIFETIME
Command Line Argument: --db-conn-max-lifetime
Default: 5m

Maximum time a database connection may be reused before it is closed and replaced. The value is specified using Go duration format, examples: 5m, 10m, 1h.

Connection Max Idle Time

Environment Variable: FILEBIN_DATABASE_CONN_MAX_IDLE_TIME
Command Line Argument: --db-conn-max-idle-time
Default: 1m

Maximum time a database connection may sit idle before it is closed. The value is specified using Go duration format, examples: 1m, 5m, 10m.

S3 Storage

S3 Endpoint

Environment Variable: FILEBIN_S3_ENDPOINT
Command Line Argument: --s3-endpoint
Default: (required)

The S3 endpoint to connect to. This can be the hostname or IP address. When self hosting S3 on a non-standard port, the port can be specified using hostname:port.

S3 Bucket

Environment Variable: FILEBIN_S3_BUCKET
Command Line Argument: --s3-bucket
Default: (required)

The name of the bucket in S3 where files will be stored.

S3 Region

Environment Variable: FILEBIN_S3_REGION
Command Line Argument: --s3-region
Default: (required)

The S3 region where the bucket lives.

S3 Access Key

Environment Variable: FILEBIN_S3_ACCESS_KEY
Command Line Argument: --s3-access-key
Default: (required)

The access key to use when connecting to the S3 bucket where files will be stored.

S3 Secret Key

Environment Variable: FILEBIN_S3_SECRET_KEY
Command Line Argument: --s3-secret-key
Default: (required)

The secret key to use when connecting to the S3 bucket where files will be stored.

S3 Secure

Environment Variable: FILEBIN_S3_SECURE
Command Line Argument: --s3-secure
Default: true

Whether or not Filebin will require the connection to S3 to be TLS encrypted using https. If set to false, Filebin will attempt connecting to S3 using plain http.

S3 URL TTL

Environment Variable: FILEBIN_S3_URL_TTL
Command Line Argument: --s3-url-ttl
Default: 1m

When a Filebin user downloads a file, that is done using a presigned URL that contains a token with limited time to live. The default allows presigned URLs to be used for 1 minute before they expire. The value is specified using the time unit, examples: 30s, 5m, 2h.

S3 Timeout

Environment Variable: FILEBIN_S3_TIMEOUT
Command Line Argument: --s3-timeout
Default: 30s

Timeout for quick S3 operations (delete, head, stat).

S3 Transfer Timeout

Environment Variable: FILEBIN_S3_TRANSFER_TIMEOUT
Command Line Argument: --s3-transfer-timeout
Default: 10m

Timeout for S3 data transfers (put, get, copy).

Lurker (Background Jobs)

Lurker Interval

Environment Variable: FILEBIN_LURKER_INTERVAL
Command Line Argument: --lurker-interval
Default: 300

The lurker is a batch job that runs automatically and in the background to delete expired bins and remove old log entries from the database. This specifies the time in seconds for the lurker to sleep between each execution.

Lurker Throttle

Environment Variable: FILEBIN_LURKER_THROTTLE
Command Line Argument: --lurker-throttle
Default: 250

Milliseconds to wait between S3 deletions to throttle the deletion rate.

Log Retention

Environment Variable: FILEBIN_LOG_RETENTION
Command Line Argument: --log-retention
Default: 7

The number of days to keep transaction log entries in the PostgreSQL database before they are removed by the lurker.

Admin Authentication

Admin Username

Environment Variable: FILEBIN_ADMIN_USERNAME
Command Line Argument: --admin-username
Default: (not set)

Username to require for access to the /admin endpoint. If the username is not set, then the admin endpoint will not be available.

Admin Password

Environment Variable: FILEBIN_ADMIN_PASSWORD
Command Line Argument: --admin-password
Default: (not set)

Password to require for access to the /admin endpoint. If the password is not set, then the admin endpoint will not be available. Make sure to keep this password a secret.

Metrics

Enable Metrics

Environment Variable: FILEBIN_METRICS
Command Line Argument: --metrics
Default: false

Enables the /metrics endpoint. If this is not set, the endpoint will not return any metrics.

Metrics Username

Environment Variable: FILEBIN_METRICS_USERNAME
Command Line Argument: --metrics-username
Default: (not set)

The username used for authentication to the /metrics endpoint for Prometheus metrics.

Metrics Password

Environment Variable: FILEBIN_METRICS_PASSWORD
Command Line Argument: --metrics-password
Default: (not set)

The password used for authentication to the /metrics endpoint for Prometheus metrics.

Metrics Auth

Environment Variable: FILEBIN_METRICS_AUTH
Command Line Argument: --metrics-auth
Default: (not set)

Enables authentication. Currently only basic auth is supported. If set to basic, basic auth will be required. If not set, the endpoint is open to the world.

Metrics ID

Environment Variable: FILEBIN_METRICS_ID
Command Line Argument: --metrics-id
Default: $HOSTNAME

The string used as the identification of the filebin instance in the Prometheus metrics. By default, this is the $HOSTNAME environment variable.

Metrics Proxy URL

Environment Variable: FILEBIN_METRICS_PROXY_URL
Command Line Argument: --metrics-proxy-url
Default: (not set)

When set, Filebin will fetch the content from the URL specified and merge it with its own output on the /metrics endpoint. This can be useful when running another Prometheus exporter in the same operating system instance, for example to capture system metrics.

Slack Integration

Slack Secret

Environment Variable: FILEBIN_SLACK_SECRET
Command Line Argument: --slack-secret
Default: (not set)

The secret that Slack will need to use when connecting to the http api. If this secret is not set, the Slack integration will be disabled.

Slack Domain

Environment Variable: FILEBIN_SLACK_DOMAIN
Command Line Argument: --slack-domain
Default: (not set)

Limits which Slack domain is allowed to access the http api. Other domains will be rejected.

Slack Channel

Environment Variable: FILEBIN_SLACK_CHANNEL
Command Line Argument: --slack-channel
Default: (not set)

If Filebin is set to require manual approval of new bins, then this approval can be given using the user interface, the http api directly or via Slack (using the http api). The http endpoint /integration/slack can be accessed using a webhook from Slack. This parameter limits which Slack channel is allowed to access the http api. Requests from other channels will be rejected.

Development

Local development environment

The development environment consists of one PostgreSQL instance, one Stupid Simple S3 object storage instance and an instance of filebin2 built from source. The easiest way to set up this environment is to clone this repository and do:

# With Docker
docker compose up --build
With Podman
podman compose up --build

This will make:

Filebin2 available on http://localhost:8080/.
Filebin2 admin available on http://admin:changeme@localhost:8080/admin.
Stupid Simple S3 available on http://localhost:5553/.
PostgreSQL available on localhost:5432.

Testing

The easiest way to run the test suite is to run it in docker compose. Docker will exit successfully (return code 0) if the tests succeed, and exit with an error code other than 0 if the tests fail.

# Docker
docker compose -f integration-tests.yml up --abort-on-container-exit
Podman
podman compose -f integration-tests.yml up --abort-on-container-exit

Building

The program can be built using:

# Build for Linux amd64 only
make linux
Build for all platforms:
make build-all

The output will be the Filebin program as binaries in the artifacts/ folder called filebin2-linux-amd64 (depending on the platform). The filebin binary take the following environment variables and command line parameters.

Integrations

Grafana and Prometheus

Filebin2 comes with a /metrics endpoint that is compatible with Prometheus. There is an example dashboard that visualizes this data.

!Dashboard

Slack

This integration may be useful if manual approval is required (see --manual-approval). The integration allows members of a Slack channel to list the recently updated bins and approve specific bins directly in the Slack channel using slash commands. The slash commands available are:

| Slash command | Description | | ------------- | ----------- | | /filebin approve bin_id | Approve the bin bin_id | | /filebin lastupdated | List the 10 last updated bins | | /filebin lastupdated n | List the n last updated bins |

The documention on how to configure Slack to work with this integration does not exist currently.

espebra/filebin2: Filebin is a web application that facilitates convenient file sharing over the web.

Table of contents