I can sincerely say, with no risk of exaggeration, that working on Hive projects over the past year has been a crazy journey. Feeling uncomfortable, sometimes unqualified, has been the common theme throughout — and yet, the work is never truly done.
For a while now, ideas have been put forward on how to improve uploads on 3Speak and speed things up. It’s no secret that the process is slow, and although we’ve tweaked and tweaked the levers, it’s still not quite there. The embed system — the video service used for Shorts — relies on a different upload approach that, while making the codebase more maintainable, slowed the process down even further.
The obvious solution for those familiar with this space is to remove IPFS from the beginning of the chain. Instead, you use the now-standard S3 technology everyone and their mother relies on, and leave IPFS at the end for long-term storage, so to speak.
It’s a good idea. A very good one, actually.
But it would also require a refactor of the sort that can age a man five years in a matter of weeks. I’m not entirely convinced I have the stamina for that battle at the moment.
With all that said, my experience with S3 buckets wasn’t exactly extensive to begin with. I’ve used them — specifically Wasabi — but I never really looked under the hood, so to speak.
Today marks a bit of a turning point, because I’ve been playing around with MinIO. I successfully deployed an instance and, what do you know, the darn thing works well.
I wish I could say the foundational rock for the future was laid today, but I’m now — and I mean this literally — beginning to understand just how much more needs to be built for a transition like this.
The best analogy I can give is this:
I got the engine to turn on. It works well. It takes fuel and converts it into torque.
But it’s hardly a car.
Multi-tenancy, statistics, load distribution, backups, scaling — none of those challenges are remotely solved yet.
It’s easy to look at the mountain of work ahead, the sheer size of the challenge, and say: “Not me.”
But I’m not insane because I want to be. It’s a feature and a bug all rolled into one.
So I’m going to start building something around MinIO.
Do we need it? Maybe not. Maybe yes. But it also can’t hurt, and I’m certain I’ll learn a ton along the way.
Madness and happiness for dinner…
What can I say?
MenO