BBolt Backed fs.FS for Fun and Profit
Alright, alright, there is no profit to be made here but it is still a neat thing you can easily achieve in Go that I wanted to share.
The Problem
This very blog you're reading didn't support any multi-media content for quite some time, which is mainly due to the fact that I couldn't get myself to use some off-the-shelf blogging software but had to write it myself. Long story, short, it works by regularly checking out a Git repository and parsing a contained .org file that is then transformed into HTML pages during run-time and stored in a BBolt database.
So how are we going to bolt on (pun intended) support for images?
Let's stick with the Go standard library, then we can use http.FileServer or the new http.FileServerFS to offload most of the logic (I already use this for serving static assets like fonts & CSS). Which leaves us with another question - where do we get a fs.FS from to pass into the file server functions?
Option A
We use os.DirFS to open a whole directory from disk as fs.FS but according to the docs, this still opens us up to path traversal attacks:
DirFS is therefore not a general substitute for a chroot-style security mechanism when the directory tree contains arbitrary content.
This makes os.DirFS unsuitable as I run the blog on bare-metal and also don't feel like messing around with chroot and creating a separate user for each hobby project I'm running
Option B
B like BBolt.
When squinting a bit, BBolt kinda looks like a limited version of a file system. So what if we just implement all the methods required by fs.FS on top of it and call it a day?
Well, that's what I did. fs.FS actually only has one method to implement which wasn't too hard:
type FS interface {
Open(name string) (File, error)
}But on top of this our returned data has to also implement fs.File and fs.FileInfo, which have quite a bunch of methods combined but most of them can return hard-coded values:
type File interface {
Stat() (FileInfo, error)
Read([]byte) (int, error)
Close() error
}
type FileInfo interface {
Name() string // base name of the file
Size() int64 // length in bytes for regular files; system-dependent for others
Mode() FileMode // file mode bits
ModTime() time.Time // modification time
IsDir() bool // abbreviation for Mode().IsDir()
Sys() any // underlying data source (can return nil)
}After that is done, we can just wrap our bbolt.DB in our new-fangled BoltFS type and slot it right into the http.FileServerFS function. Pretty convenient.
Advantages
A big pro definitely is that the whole thing is quite simple. It reuses our BBolt instance we already had lying around and the standard library takes care of the rest, so we even get automatic ETag support out of the box.
Security-wise, I believe at least, that it is quite safe because you can't access anything that is not in a specific bucket in our BBolt database, so a malicious client shouldn't be able to poke around on our Linux server.
Limitations
Serving large files this way is not a good idea as the whole file will be loaded into memory on every request. This will probabaly also strain the garbage collector quite a bit but for the non-existent traffic I have it will do.
ETag-based caching also won't work correctly because the modification date is hard-coded. To properly function we would need to store the current time alongside every value in our database like so
type BBoltFile[T any] struct {
Data T
Modified time.Time
}or something along those lines. But I currently don't.
Verdict
It is always good fun to misuse one component as something else and it was also pretty straight forward to get something to work even though the current solution is not quite optimal, yet.
So please forgive me for wasting your network bandwith with a bad caching implementation and the excessive use of memes.
