A fast, auto-optimizing image server designed for high throughput and caching; Now that is hot.
Go to file
Harrison Burt 91c6becfbd adjust for conversion types 2021-07-04 21:29:39 +01:00
.github Update auto-build.yaml 2021-07-04 19:57:53 +01:00
assets fix enabling and disable conversions 2021-05-28 21:38:48 +01:00
src correct error 2021-07-04 19:14:34 +01:00
tests add timings to debug logs to track format conversion speed. 2021-07-04 19:09:34 +01:00
webp adjust for conversion types 2021-07-04 21:29:39 +01:00
.dockerignore dont think chmod is really needed 2021-06-05 22:01:15 +01:00
.gitignore add custom webp library 2021-06-18 14:45:50 +01:00
Cargo.toml bump lust version 2021-07-04 21:19:48 +01:00
Dockerfile adjust dockerfile 2021-07-04 18:53:55 +01:00
LICENSE Create LICENSE 2021-05-28 19:16:38 +01:00
README.md update dockerfile in readme 2021-06-06 16:38:09 +01:00
getting-started.md update docs 2021-07-04 19:56:00 +01:00

README.md

Lust Logo

What is Lust?

Lust is a static image server designed to automatically convert uploaded image to several formats and preset sizes with scaling in mind.

Lust stores images via any of given database backends:

  • Redis / KeyDB
  • Cassandra / ScyllaDB
  • PostgreSQL
  • MySQL / MariaDB
  • Sqlite (file / temp file only)

Getting started

Installation

Building from Source

To building from source, just clone this repo via git clone https://github.com/chillfish8/lust.git and then run cargo build --release.

Docker Images

Lust has a set of pre-built, optimised docker images ready to go, they just require having a config.json attached to them and away you go.

Example Dockerfile:

FROM chillfish8/lust:latest

ADD ./config.json /etc/lust/config.json

You can run the image via docker run, you may wish to expose your set ports etc...

After Installation

See the getting started page for more information after installation.

Caching

Lust makes use of a Least Recently Used in-memory cache which can be adjusted for your needs via the cache_size key in the configuration file. The larger the number the more images it will cache at once and vice versa. NOTE: With bigger images this can create much higher RAM usage

Scaling

Lust's ability to scale is purely down to the backend you use, something like SQLite will obviously suffer at any sort of scale and is meant only really for development purposes. Personally I recommend PostgreSQL (leading to vertical scaling storage) or Scylla (Horizontally scaling storage) depending on your needs. If you want a very small amount of cached images then Postgres will out perform Scylla considerably at random reads however, Scylla is far more suites to large scaling and distributed system as well as large amounts of writes.

Performance of each database generally doesn't matter too much due to the processing time of each image being more than the IO latency when adding images and the cache supporting reads, that being said if you have a lot of random inconsistent reads PostgreSQL will likely be the best, or if you want large distributed scaling Scylla will allow you to scale horizontally.

If you want the best of both worlds I would recommend looking at KeyDB (Redis) with disk persistence, when setup correctly this can be an incredibly powerful setup.

Formats

Lust supports any of the following formats:

  • Png
  • JPEG
  • GIF
  • Webp

Any uploaded images will be given a unique uuid and be re-encoded into all the other enabled formats in all presets. This is especially useful when you want to serve several variants of the same image with different formats.

Presets

The server can take several sizing presets which can be targeted via the size query parameter when getting an image. These presets will mean every image at upload time will be resized to fit the width and height bounds using the nearest approximation.

Regardless of presets an original image is always stored and can be accessed via the size=original query. The default preset when served without a sized parameter can be set in the configuration file via default_serving_preset key.

Webp Optimisation

Lust supports automatic webp encoding, by default it encodes with lossless compression but this can be changed via the webp_quality key in the configuration file and should be a float from 0.0 to 100.0 with the quality of the image changing respectively.

Base64 Support

Lust will serve given images / gifs as Base64 data the encode query parameter (true/false) this will return a JSON response unlike the tradition raw response.

Data Efficiency

Lust's data storage efficiency is roughly the same as storing on a plain file system outside of any system the database backend employs when storing the data.

For example lets upload an image:

Medium image

This image is about 91KB in size as a single image, If we upload this and convert to the 3 base image formats with some presets:

{
  'data': {
    'file_id': 'ccbe2207-8629-4938-9da9-3f75706f9b4e',
    'formats': {
      'large': {     // Resized to 128px x 128px
        'jpeg': 3460,
        'png': 5292,
        'webp': 3006
      },
      'medium': {   // Resized to 64px x 64px
        'jpeg': 1543, 
        'png': 1738, 
        'webp': 1022
      },
      'original': {   
        'jpeg': 42846,
        'png': 103672, 
        'webp': 53982
      },
      'small': {    // Resized to 32px x 32px
        'jpeg': 912, 
        'png': 629, 
        'webp': 354
      }
  },
  'status': 200
}

We can see with the original size totals around 200KB which is is fairly reasonable with zero compression PNG encoding and lossless webp formats.