Minio: self-hosted S3-compatible object store in Docker
Posted: Wed Jul 26, 2023 3:50 pm
I've been messing with Wagtail CMS and like it. It's based on Django, a monolithic Python app framework that includes a CMS and massive ecosystem of "apps" that are composed together. By default uploaded files are written to disk, but that's no good for "serverless" deployments like Lambda or Vercel that run app code but don't have local disk persistence.
That's where the need for an object store like S3 comes in. Vercel has a waitlist for their S3-compatible "Blob Store," which would fit the bill. I've already validated their django-starter template with the Postgres to back the ORM and the "KV"/Redis-compatible offering for page cache and it works fine. But I'm on the waitlist and they didn't give me any special treatment for Tweeting at them a couple of times what I'm doing.
So like the self-hosting enthusiast I am, I took it into my own hands. I researched S3-compatible options and went to work. I first tried to find options that will run in Glitch, my go-to free hosting environment. The terminal access is great but can be quite limiting when you're not sure of the outcome. I wasn't sure about getting this or that package to run in their environment. Thankfully I've been getting work lately and can invest a little bit in a cloud server and domain name after some time without (for pretty much the first time in my life).
The first offering I tried was an old Python 2.7 package for mocking S3, jserver/mock-s3. I actually got it to list buckets with boto3, but for some reason I feel like it's going to break when I hook it up to wagtail-storages. We can back up and try this once I am confident that wagtail works with something more robust. The name of the game I'm most comfortable with is "get something working and then cut costs,"--aka. time to market is king.
When operating without money a person has time, so I'd spend days or weeks trying to validate these things. With a few dollars to invest in a small VPS/cloud server I can just try the titanic industrial grade platform. I've used something like Riak before when investigating on on-prem situation for a system designed and built in AWS with S3. Minio is new but S3-compatibility means there will be some level of predictability with buckets and policies, and there was. I got everything I expected with Minio, quite pleased with it and how polished the Console is.
One slick thing I like is the sharing URL. It only goes for up to 7 days, so after that this post will break... but hopefully before then I'll figure out how to get this PhpBB installation wired up to S3, I've already found an extension to do it. Perhaps tricking out PhpBB will be another Dev Journal entry, stay tuned.
Here's the steps I took. I'm writing for people who are proficient in DevOps already, apologies I don't have the manpower to go in and polish it all up. I mention how to ask questions at the end, and I can update this post with more details as/if requested. Thanks for understanding.
The steps I took to deploy the system were:
I went with the Bitnami one: bitnami/minio
Configure and run the Docker image
Wire up an Nginx proxy
This doesn't include SSL which will be handled with the next step. The file goes in and needs a symlink in to run. Check with .
Run letsencrypt/certbot to get SSL on the proxy
As root or with sudo run the following command and
.
Set up a couple users and attached policies to them
Skipping some steps, the convention I went with was creating a couple users for my projects and giving them free reign to a namespace of buckets.
For each user I create a policy called like:
And then for each app I can create an access key with further restrictions, like
Which is used for the notes-app and gives it full access only to that particular bucket. A public read/write-only or similar could be created for portions of the bucket if needed, etc... that's where this tutorial ends and your standard S3 guides need to pick up.
Hopefully this was helpful. Obviously a lot of details are glossed over, assuming you have knowledge of running docker, setting up DNS records, and ability to find help for the command line options I used. I have an Ask HarlanJI section that can be used without registration if you have any questions, or you can register and if I know you I'll approve it.
That's where the need for an object store like S3 comes in. Vercel has a waitlist for their S3-compatible "Blob Store," which would fit the bill. I've already validated their django-starter template with the Postgres to back the ORM and the "KV"/Redis-compatible offering for page cache and it works fine. But I'm on the waitlist and they didn't give me any special treatment for Tweeting at them a couple of times what I'm doing.
So like the self-hosting enthusiast I am, I took it into my own hands. I researched S3-compatible options and went to work. I first tried to find options that will run in Glitch, my go-to free hosting environment. The terminal access is great but can be quite limiting when you're not sure of the outcome. I wasn't sure about getting this or that package to run in their environment. Thankfully I've been getting work lately and can invest a little bit in a cloud server and domain name after some time without (for pretty much the first time in my life).
The first offering I tried was an old Python 2.7 package for mocking S3, jserver/mock-s3. I actually got it to list buckets with boto3, but for some reason I feel like it's going to break when I hook it up to wagtail-storages. We can back up and try this once I am confident that wagtail works with something more robust. The name of the game I'm most comfortable with is "get something working and then cut costs,"--aka. time to market is king.
When operating without money a person has time, so I'd spend days or weeks trying to validate these things. With a few dollars to invest in a small VPS/cloud server I can just try the titanic industrial grade platform. I've used something like Riak before when investigating on on-prem situation for a system designed and built in AWS with S3. Minio is new but S3-compatibility means there will be some level of predictability with buckets and policies, and there was. I got everything I expected with Minio, quite pleased with it and how polished the Console is.
One slick thing I like is the sharing URL. It only goes for up to 7 days, so after that this post will break... but hopefully before then I'll figure out how to get this PhpBB installation wired up to S3, I've already found an extension to do it. Perhaps tricking out PhpBB will be another Dev Journal entry, stay tuned.
Here's the steps I took. I'm writing for people who are proficient in DevOps already, apologies I don't have the manpower to go in and polish it all up. I mention how to ask questions at the end, and I can update this post with more details as/if requested. Thanks for understanding.
The steps I took to deploy the system were:
- Find a Minio Docker image
- Configure and run the Docker image
- Create a CNAME DNS record
- Wire up an Nginx proxy
- Run letsencrypt/certbot to get SSL on the proxy
- Play around with the running installation
- Blow away the running installation and restarted it with tweaked params
- Set up a couple users and attached policies to them
I went with the Bitnami one: bitnami/minio
Configure and run the Docker image
Code: Select all
docker volume create minio-data_minio.harlanji.com
docker run --name minio_minio.harlanji.com \
-d \
-p 127.0.0.1:9000:9000 -p 127.0.0.1:9001:9001 \
-e MINIO_SERVER_URL="https://minio.harlanji.com" \
-e MINIO_BROWSER_REDIRECT_URL="https://minio.harlanji.com/console/" \
-e MINIO_ROOT_USER="___________" \
-e MINIO_ROOT_PASSWORD="___________" \
-v minio-data_minio.harlanji.com:/data \
bitnami/minio:2023.7.18-debian-11-r2
Code: Select all
server {
server_name minio.harlanji.com;
# Allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# Disable buffering
proxy_buffering off;
proxy_request_buffering off;
location /console {
rewrite ^/console/(.*) /$1 break;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websockets in MinIO versions released after January 2023
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://localhost:9001;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://localhost:9000;
}
listen 80;
listen [::]:80;
}
Code: Select all
/etc/nginx/sites-availableCode: Select all
/etc/nginx/sites-enabledCode: Select all
nginx -tRun letsencrypt/certbot to get SSL on the proxy
As root or with sudo run the following command and
Code: Select all
certbot --nginx
Set up a couple users and attached policies to them
Skipping some steps, the convention I went with was creating a couple users for my projects and giving them free reign to a namespace of buckets.
For each user I create a policy called
Code: Select all
harlanji-bucketsCode: Select all
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::harlanji.*",
"arn:aws:s3:::harlanji.*/*"
]
}
]
}
Code: Select all
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::harlanji.notes-app",
"arn:aws:s3:::harlanji.notes-app/*"
]
}
]
}
Hopefully this was helpful. Obviously a lot of details are glossed over, assuming you have knowledge of running docker, setting up DNS records, and ability to find help for the command line options I used. I have an Ask HarlanJI section that can be used without registration if you have any questions, or you can register and if I know you I'll approve it.