About a week ago I noticed that the memo messages that you can attach to a transaction can hold a lot more information that what you would expect.
You can store about 2048 characters per memo. that's 2 kb. If you don't really know what this represents, well, so far I've only written 286 characters.
Right now you're probably like "yeah 2048 chars is nice but posts can hold so much more". I actually don't know the size limit of a post, but I know that you can only send one post every 5 minutes, and that's just not gonna cut it. Memos don't have such limitations which gives us a lot more flexibility.
What is a file :
Another thing you might say is : "Yeah but memos can only hold text information". I'll let you in on a little secret :
Files are nothing but characters
If you open with an image it will probably look like this :
Lots of random (and often unreadable) characters. But it's actually just information about the image, it's looking like that because most image format are compressed so we try to use as little information as possible to represent the pixels. If you open a .pdf you can notice that it's already way more structured :
So what software does when it reads a file is simply read it and parse the information on it to produce the desired effect, an image viewer will read the pixels infos and display them, a pdf reader will display the pdf etc. You could produce a pdf reader if you want, you just have to know the format of the file and write a program that does that.
Memos :
To write data to memos we need to store it as readable characters, so we use base64 to encode the data. Base64 is made specifically for the purpose of representing binary data with readable characters so it's perfect.
Since a memo holds 2kb of data, to store 100kb, we'll need around 50 memos.
The storing scheme is very simple :
File -> binary data -> base64 encoded data -> split in x subtexts of 2kb -> send to memos
When we send the memos, we retrieve its unique id and store it. And when the complete file is uploaded to the blockchain, we send one last memo that holds the ids of all the previous memos and the filename.
It looks like this : "data/steem.png:3294:3295:3296:3297:3298:3299:3300:3301:3302:3303:3304:3305"
If you're curious you can see it on my steemd or if you're reading this a long time in the future :
Retriving the data
Retrieving the data is pretty easy, we just look backwards (starting from the most recent memo). For the filename.
Once we find the filename, we parse the header file to know on which memos the data is stored. We read them, put them together again, decode using base64 and then write the binary data onto a file. And bam ! you got your file back.
Since we look for the most recent memo header, if you write a file named "steem.png" and then another file with the same name, we will retrieve the first one.
Conclusion
Obviously this puts your data out in the open for anyone to see, so don't put anything that you would not put in public.
Since we are sending transfers to ourselves, you can store data with 0.001 sbd. This also means that an attacker needs your active key to write on your storage. And if someone got your active key, you got bigger problems than that.
The only limit is your bandwidth. Right now with 1067 sp I got 1.37gb, so I can store quite a lot of data on there, It would be interesting to see if we could write a front end that writes the images to the blockchain. Because then we would have completely decentralized posts. that can be red by anyone, anywhere, with anything.
Anyway, it's been very fun to work with memos, I will definitely experiment a bit more with this idea and try to implement a database.
That would allow some services to work completely on-chain without any centralization. Perhaps even some form of very basic smart contract where bots parse the memo code and can interact with it to change it and execute its code.
I will also look into encrypted memos. This would provide privacy to our storage but it's pretty risky because if someones manages to get your memo key then everything that you stored becomes available to everyone and you have can't erase it.
If you're curious, you can find the code here : https://github.com/drov0/MemoBox
Posted on Utopian.io - Rewarding Open Source Contributors