What Will I Learn?
- Basics of JavaScript syntax, Node.js projects, and
npmusage - Use Microsoft VSCode as IDE
- Grab HTML data from Internet web pages
- Use Cheerio to parse, query and manipulate DOM tree from HTML
- Basic usage of
steemit-jslibrary - Use Cron and Node.js
cronlibrary to schedule tasks
Stemmit provides APIs for us to do a lot of awesome things -- this is what makes it the best blogging platform in my mind. Now, how to use them to develop our own bot? Just follow my step.
Which Programming Language?
Well, this is also the best part of it: not only did Steemit provide APIs, but also in so many popular programming languages. So, how to choose?
Basically, choose which ever you are good at.
My first choice was Python. But its support for Windows is not that good (I know Windows is not the best for developing, but I kind of need it so much, and I cannot successfully install the steem-python package there, at all). So I just go to JavaScript.
So this tutorial shows a JavaScript example
Anything to Prepare?
Yes, a JavaScript engine is required to run you code. Any browser can do it, but it's not convenient at all.
A. Node.js is a good recommendation to run the JavaScript project.
To Install Node.js, check here to find whatever suits you OS.
B. A good IDE makes things a lot easier, although it's optional.
IDE refers to "Integrated Development Environment", a piece of software to help you write, debug and manage your project codes. Here I am using Microsoft's VSCode to organize project code. It natively supports Node.js projects, including GIT stuffs. If you need more functionality such as Python support or TODO parser, just search relative extensions. VSCode support plenty of such kind of extensions, which cover almost any need for any programmers.
Shall We Start?
Of course, let's do it!
A. Setup the shell Node.js project -- "SteemitBlogger"
- Open the VSCode, and open the root directory of the project, let's say,
D:\Projects\SteemitBlogger\.
- Now press
Ctrl+`(the one above CapsLock and Tab) to show up the terminal panel, runnpm initin it, and follow the guide to type in the information, including package name, version, etc. These operations created thepackage.json. In the example I accepted all defaults.
- In the
package.json, the entry was default set to beindex.js. This file needs to be manually created, and it's where we code the blogger.
B. Coding I: what to write in the blog
A Steemit blog contains a title, an author, several keywords as tags, and a body. These are the things we should prepare. Remember this is an autobot, composing and publishing blogs repeatedly. So you want different titles and bodies, maybe different tags, too. Let's do it like this:
// put date into the fields so they are different each day
var now = new Date().toISOString(); // the current time in UTC time zone, in format of "2017-12-31T23:59:59Z",
// where the trailing 'Z' refers to the UTC time zone.
var title = 'Hello World on ' + now.split('T')[0]; // this 'split' separates the time string by 'T', and takes
// the first part (JavaScript lists start with index 0)
var author = 'user1';
var body = 'Hello to everyone in the world at' + now + '\n\n$PLACE_HOLDER\n';
I want fill something later to the body, so I define the above $PLACE_HOLDER for now.
C. Coding II: grab data from Internet and put them in
Sure. Let's say, I want to find what Microsoft did in GitHub, i.e., the repository names.
The link involved is https://github.com/Microsoft.
C.1 Get the html data and parse DOM
DOM refers to "Document Object Model", which stores all information of a web page into a tree structure, with standard APIs to find and modify information in it. We use Cheerio to parse HTML strings to DOM trees, it operates the same way as jQuery. It's OK if you don't know Cheerio/jQuery, just follow my example.
To install Cheerio, type the following command in terminal:
npm install --save cheerio # '--save' writes cheerio into package.json dependency records
Above shows a short version of the install command. After it, dependency of cheerio appended to package.json, and a folder node_modules added to the project containing all dependencies.
Node.js provides http.get and https.get to read data from web pages. We just use it like this:
var cheerio = require('cheerio');
var https = require('https');
function grabData() { // Put this job in a function
https.get('https://github.com/Microsoft', function(res) { // The 'get' procedure is standard,
res.on('error', function(err) { // so the code snippet applies all similar cases
throw err;
}); // res.on('error', function(err) { ... });
var html = '';
res.on('data', function(data) {
html += data;
}); // res.on('data', function(data) { ... });
res.on('end', function() {
var $ = cheerio.load(html); // This line parses HTML string to DOM with root '$'
// Do something with the DOM
// ... ...
}); // res.on('end', function() { ... });
}); // https.get('https://github.com/Microsoft', function(res) { ... });
} // function grabData() { ... }
C.2 Ming the DOM
Use Google Chrome's (similar with other browsers) Developer Tools, we can dig DOM information directly.
Just press F12 and follow this:
In the Developer Tools, choose "Elements" tab, and toggle the left top button on, then you can quickly navigate the corresponding
DOM element when you hover the mouse cursor on anything in the web page. We can see each of the target text locates in an <a> with the attribute itemprop value of name codeRepository, so we grab such data in the following way:
// ... ...
// Do something with the DOM
var data = '';
$('[itemprop="name codeRepository"]') // use '[attr="value"]' to select all element with target attribute/value
.each(function(i, e) { // '.each' iterates every selected element
data += ' + ' + e.children[0].data.trim() + '\n'; // the texts are stored in its first child's 'data' field
}); // $( ... ).each( ... );
body = body.replace('$PLACE_HOLDER', data); // put the data to the correct place in body
// ... ...
C.3 Prepare and publish the blog to Steemit
As been said, a Steemit blog contains title, author, tags and body, but they are not all. The full list contains:
title: simply the above title;author: simply the above author;body: simply the above body, in markdown format (see below);permlink: a permanent link for this blog, could be concatenated by author and title, with spaces being properly dealt with:// Prepare and publish the blog // '/\s+/g' is a regular expression to match all continuous spaces var permlink = (author + '-' + title).replace(/\s+/g, '-');json_metadata: this is a JSON object that contains tags, for Steemit blogs, it could always be set to:var json_metadata = { "tags": ["tag1", "tag2", "tag3", "tag4"], // the tag list you want add "app": "steemit/0.1", // always be like this "format": "markdown" // as mentioned above, body in markdown format }; // var json_metadata = { ... };tags: they are included in json_metadata; and of courseposting_key: you need this to publish the blog, it must matchauthor.
steem.broadcast.comment is the function to publish it. So you need steem-js library:
npm i -S steem@0.6.4 # Specify the stable version of steem-js
With the above parameters, you publish a blog like this:
var steem = require('steem');
var posting_key = 'user1\'s-posting-key';
steem.broadcast.comment(
posting_key, // must be the private key that matches the author
'', // parent's author. Blogs to be empty string, replies to be the blog replied to
'tutorial', // parent’s permlink. Blogs to be the category; replies to be the blog replied to
author, // Steemit user name
permlink, // permanent link of this blog
title, // blog title; replies to be empty string
body, // blog body in markdown format; replies to be the reply message
json_metadata, // json_metadata contains tags; replies could be empty string
function(err, re) {
if (err) {
throw err;
} // if (err)
}); // steem.broadcast.comment( ... );
D. Coding III: schedule and repeat the fantastic job*
We use cron to schedule it:
npm i -S cron
It uses a string in the form of 6 fields: minute, hour, day of month, day of week, month, day of week. In each field, * means match every value, */5 means match every 5 times, and a specific value means match the value. So 00 05 01 * * * means match every day at 01:05:00AM. This is what we want -- to repeat everyday.
var CronJob = require('cron').CronJob;
new CronJob(
'00 05 01 * * *', // set the job to repeat at 01:05:00AM everyday
function() { // what to do at that time
grabData();
},
null, // what to do when stopped -- nothing
true, // shall we start the job right now? Of course
'UTC' // 01:05:00AM of which time zone
); // new CronJob( ... );
E. A need-to-know
Add this line to change the URL of steem API, otherwise when the task repeats an error will be raised by steem, causing the program crashed.
steem.api.setOptions({url: 'https://api.steemit.com'});
F. Run it
Go to menu Debug=>Add Configuration ..., then choose Node.js in the quick panel, the file .vscode/launch.json will be created automatically storing the launching configurations.
Now press F5 to run it. After running, press F6 to pause it, Shift+F5 to stop it, and Ctrl+Shift+F5 to restart it.
To Conclude?
Nothing. Just check the full code, and remember to do npm i for the first run. Wish you good luck! :)
// ---- package.json ----
{
"name": "steemitblogger",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"cheerio": "^1.0.0-rc.2",
"cron": "^1.3.0",
"steem": "^0.6.4"
}
}
// ---- End of package.json ----
// ---- index.js ----
var cheerio = require('cheerio');
var CronJob = require('cron').CronJob;
var https = require('https');
var steem = require('steem');
// put date into the fields so they are diffrent each day
var now = new Date().toISOString(); // It's the current time in UTC timezone, in format of "2017-12-31T23:59:59Z",
// where the trailing 'Z' refers to the UTC timezone.
var title = 'Hello World on ' + now.split('T')[0]; // this 'split' seperates the time string by 'T', and takes
// the first part (JavaScript lists start with index 0)
var author = 'user1';
var body = 'Hello to everyone in the world at' + now + '\n\n$PLACE_HOLDER\n';
function grabData() { // Put this job in a function
https.get('https://github.com/Microsoft', function(res) { // The 'get' procedure is standard,
res.on('error', function(err) { // so the code snippet applies all similar cases
throw err;
}); // res.on('error', function(err) { ... });
var html = '';
res.on('data', function(data) {
html += data;
}); // res.on('data', function(data) { ... });
res.on('end', function() {
var $ = cheerio.load(html); // This line parses HTML string to DOM with root '$'
// Do something with the DOM
var data = '';
$('[itemprop="name codeRepository"]') // use '[att="value"]' to select all element with target attribute/value
.each(function(i, e) { // '.each' iterates every selected element
data += ' + ' + e.children[0].data.trim() + '\n'; // the texts are stored in its first child's 'data' field
}); // $( ... ).each( ... );
body = body.replace('$PLACE_HOLDER', data); // put the data to the correct place in body
// Prepare and publish the blog
// '/\s+/g' is a regular expression to match all continuous spaces
var permlink = (author + '-' + title).replace(/\s+/g, '-');
var json_metadata = {
"tags": ["tag1", "tag2", "tag3", "tag4"], // the tag list you want add
"app": "steemit/0.1", // always be like this
"format": "markdown" // as mentioned above, body in markdown format
}; // var json_metadata = { ... };
var posting_key = 'user1\'s-posting-key';
steem.broadcast.comment(
posting_key, // must be the private key that matches the author
'', // parent's author. Blogs to be empty string, replies to be the blog replied to
'tutorial', // perent's permlink. Blogs to be the category; replies to be the blog replied to
author, // Steemit user name
permlink, // permanent link of this blog
title, // blog title; replies to be empty string
body, // blog body in markdown format; replies to be the reply message
json_metadata, // json_metadata contains tags; replies could be empty string
function(err, re) {
if (err) {
throw err;
} // if (err)
}); // steem.broadcast.comment( ... );
}); // res.on('end', function() { ... });
}); // https.get('https://github.com/Microsoft', function(res) { ... });
} // function grabData() { ... }
new CronJob(
'00 05 01 * * *', // set the job to repeat at 01:05:00AM everyday
function() { // what to do at that time
grabData();
},
null, // what to do when stopped -- nothing
true, // shall we start the job right now? Of course
'UTC' // 01:05:00AM of which timezone
); // new CronJob( ... );
// ---- End of index.js ----
All images and texts are created by 
Posted on Utopian.io - Rewarding Open Source Contributors