mirror of https://github.com/ChillFish8/lust.git
Update readme
This commit is contained in:
Harrison Burt
parent
1a1c7511ba
commit
df1f50c28e
134
README.md
134
README.md
|
|
@ -1,39 +1,45 @@
|
|||
<p align="center">
|
||||
<img width="60%" src="https://github.com/ChillFish8/lust/blob/master/assets/logo.png" alt="Lust Logo">
|
||||
<img width="50%" src="https://user-images.githubusercontent.com/57491488/160932579-518e61b8-6a3d-4400-a46c-1cb93d461417.png" alt="Lust Logo">
|
||||
</p>
|
||||
|
||||
#
|
||||
🔥 Build your own image CDN system your way with lust.
|
||||
|
||||
## What is Lust?
|
||||
Lust is a static image server designed to automatically convert uploaded image to several formats and preset sizes with scaling in mind.
|
||||
|
||||
Lust stores images via any of given database backends:
|
||||
|
||||
- Redis / KeyDB
|
||||
- Cassandra / ScyllaDB
|
||||
- PostgreSQL
|
||||
- MySQL / MariaDB
|
||||
- Sqlite (file / temp file only)
|
||||
Lust is an auto-optimising image server, designed for high throughput and low latency handling of images, *now that is lustful*.
|
||||
Re-encode uploaded images into `png`, `jpeg`, `webp` or even into `gif` based formats!
|
||||
|
||||
Resize them to your liking automatically with sizing presets, instantly create small,
|
||||
medium and large variants with just a few line in a config file. *Now that's the spirit of lust*
|
||||
|
||||
And much more like caching, on the fly resizing, processing modes to name a few.
|
||||
## Getting started
|
||||
|
||||
### Creating a config file
|
||||
It's highly advised to take a look at some [example config files](/examples/configs) to get an idea
|
||||
of what a general config file would look like.
|
||||
|
||||
Full documentation in markdown form can also be found [here](description.md), this is also
|
||||
served directly by the server as part of the documentation ui endpoint.
|
||||
|
||||
### Installation
|
||||
#### Building from Source
|
||||
To building from source, just clone this repo via `git clone https://github.com/chillfish8/lust.git` and then run `cargo build --release`.
|
||||
#### Docker Images
|
||||
Lust has a set of pre-built, optimised docker images ready to go, they just require having a config.json attached to them and away you go.
|
||||
|
||||
Example Dockerfile:
|
||||
```docker
|
||||
FROM chillfish8/lust:latest
|
||||
|
||||
ADD ./config.json /etc/lust/config.json
|
||||
#### Installing via Cargo
|
||||
You can install lust directly via cargo and the git flag:
|
||||
```shell
|
||||
cargo install lust --git https://github.com/ChillFish8/lust.git
|
||||
```
|
||||
|
||||
You can run the image via `docker run`, you may wish to expose your set ports etc...
|
||||
#### Docker Images
|
||||
Lust has a set of pre-built, optimised docker images ready to go. Just run it with
|
||||
```shell
|
||||
docker run -v "my_config.yaml:/var/lust/my_config.yaml" chillfish8/lust:latest --config-file "/var/lust/my_config.yaml"
|
||||
```
|
||||
|
||||
### After Installation
|
||||
See the [getting started page](https://github.com/ChillFish8/lust/blob/master/getting-started.md) for more information after installation.
|
||||
Once you're up and running navigate to `http://127.0.0.1:8000/ui` or `/ui` of what ever port your server is running on
|
||||
to see the full OpenAPI docs.
|
||||
|
||||
## Caching
|
||||
Lust makes use of a Least Recently Used in-memory cache which can be adjusted for your needs via the `cache_size` key in the configuration file.
|
||||
|
|
@ -41,19 +47,12 @@ The larger the number the more images it will cache at once and vice versa.
|
|||
*NOTE: With bigger images this can create much higher RAM usage*
|
||||
|
||||
## Scaling
|
||||
Lust's ability to scale is purely down to the backend you use, something like SQLite will obviously suffer
|
||||
at any sort of scale and is meant only really for development purposes.
|
||||
Personally I recommend PostgreSQL (leading to vertical scaling storage) or Scylla (Horizontally scaling storage) depending on your needs.
|
||||
If you want a very small amount of cached images then Postgres will out perform Scylla considerably at random reads however,
|
||||
Scylla is far more suites to large scaling and distributed system as well as large amounts of writes.
|
||||
Lust's ability to scale is purely down to the backend you use, so it is worth noting that
|
||||
the file system backend is only really designed for testing. For full scale deployment
|
||||
consider using Scylla or a s3 compatible blob store to serve data from.
|
||||
|
||||
Performance of each database generally doesn't matter too much due to the processing time of each image
|
||||
being more than the IO latency when adding images and the cache supporting reads, that being said if
|
||||
you have a lot of random inconsistent reads PostgreSQL will likely be the best, or
|
||||
if you want large distributed scaling Scylla will allow you to scale horizontally.
|
||||
|
||||
If you want the best of both worlds I would recommend looking at KeyDB (Redis) with disk persistence, when setup correctly this
|
||||
can be an incredibly powerful setup.
|
||||
If your goal is high-end performance, Scylla DB will be the most performant by a large
|
||||
margin, but this will come with a higher operating cost.
|
||||
|
||||
## Formats
|
||||
Lust supports any of the following formats:
|
||||
|
|
@ -64,60 +63,29 @@ Lust supports any of the following formats:
|
|||
|
||||
Any uploaded images will be given a unique uuid and be re-encoded into all the other enabled formats in all presets.
|
||||
This is especially useful when you want to serve several variants of the same image with different formats.
|
||||
|
||||
You can also adjust this based on the processing mode, `aot`/*Ahead of time* encoding will follow the old
|
||||
lust behavour by encoding and resizing each image at upload time.
|
||||
|
||||
`jit`/*Just in time* encoding will only resize and re-encode at request time, storing a base copy
|
||||
of the file to generate new images. This can save on a considerable amount of CPU time and disk space
|
||||
depending on your requirements.
|
||||
|
||||
Finally, we have the `realtime` encoder, this will only store an original copy like the `jit` encoder
|
||||
but instead will never save the resized and encoded image, this does also enable the ability to
|
||||
do on the fly resizing and is recommended for situations where you're not expecting to serve image
|
||||
to the public network.
|
||||
|
||||
## Presets
|
||||
The server can take several sizing presets which can be targeted via the `size` query parameter when getting an image. These presets will mean every image at upload time will be resized to fit the width and height bounds using the nearest approximation.
|
||||
The server can take several sizing presets which can be targeted via the `size`
|
||||
query parameter when getting an image.
|
||||
These presets will mean every image at upload time will be resized to
|
||||
fit the width and height bounds using the configured resizing filter
|
||||
(defaults to nearest neighbour).
|
||||
|
||||
Regardless of presets an `original` image is always stored and can be accessed via the `size=original` query.
|
||||
The default preset when served without a `sized` parameter can be set in the configuration file via `default_serving_preset` key.
|
||||
|
||||
## Webp Optimisation
|
||||
Lust supports automatic webp encoding, by default it encodes with lossless compression but this can be changed via the `webp_quality` key in the configuration file
|
||||
and should be a float from `0.0` to `100.0` with the quality of the image changing respectively.
|
||||
|
||||
## Base64 Support
|
||||
|
||||
Lust will serve given images / gifs as Base64 data the `encode` query parameter (`true`/`false`) this will return
|
||||
a JSON response unlike the tradition raw response.
|
||||
The default preset when served without a `size` parameter can be set in the configuration file via `default_serving_preset` key.
|
||||
|
||||
## Data Efficiency
|
||||
Lust's data storage efficiency is roughly the same as storing on a plain file system outside of any system the database backend employs when storing the data.
|
||||
|
||||
For example lets upload an image:
|
||||
<p align="left">
|
||||
<img width="50%" src="https://github.com/ChillFish8/lust/blob/master/assets/news.png" alt="Medium image">
|
||||
</p>
|
||||
|
||||
This image is about 91KB in size as a single image, If we upload this and convert to the 3 base image formats with some presets:
|
||||
|
||||
```json5
|
||||
{
|
||||
'data': {
|
||||
'file_id': 'ccbe2207-8629-4938-9da9-3f75706f9b4e',
|
||||
'formats': {
|
||||
'large': { // Resized to 128px x 128px
|
||||
'jpeg': 3460,
|
||||
'png': 5292,
|
||||
'webp': 3006
|
||||
},
|
||||
'medium': { // Resized to 64px x 64px
|
||||
'jpeg': 1543,
|
||||
'png': 1738,
|
||||
'webp': 1022
|
||||
},
|
||||
'original': {
|
||||
'jpeg': 42846,
|
||||
'png': 103672,
|
||||
'webp': 53982
|
||||
},
|
||||
'small': { // Resized to 32px x 32px
|
||||
'jpeg': 912,
|
||||
'png': 629,
|
||||
'webp': 354
|
||||
}
|
||||
},
|
||||
'status': 200
|
||||
}
|
||||
```
|
||||
|
||||
We can see with the `original` size totals around 200KB which is is fairly reasonable with zero compression PNG encoding and lossless webp formats.
|
||||
Lust's data storage efficiency is roughly the same as storing on a plain file system outside any
|
||||
system the database backend employs when storing the data.
|
||||
|
|
|
|||
Loading…
Reference in New Issue