First of all, I'm very pleased that all of my pull requests for devportal have reached the Steemit, Inc. official site, as reported yesterday. My goal was to try to keep my fork and the origin fork as 1-to-1 as possible. There's no reason for me to have unique documentation on my fork.
It looks like it took them only a couple days to fully merge my changes that spanned over a month. Great job, Steemit, Inc! I believe moving forward, this should be much easier now that the relative URL structure has been addressed.
Here, I'd like to go over the various rake tasks that have also been merged to the official repository.
Some of these tasks are shown in the 
Repository
https://github.com/inertia186/devportal
Rake (rake) is a command-line tool that means "Ruby Make." The command implements tasks defined in a Rakefile, usually in a cloned git project.
In the devportal Steem documentation project, the rake tasks are as follows:
$ rake -vT
rake ops_dump[vops,appbase] # Dump all operation types
rake production:deploy # Deploy current master to GH Pages
rake production:rollback # Rollback GH Pages
rake scrape:api_defs # Scrape API Definitions
rake scrape:javascript # Scrape steemjs docs
rake scrape:python # Scrape pysteem docs
rake scrape:tutorials # Scrape all known tutorial repositories
rake scrape:tutorials:js[num,force] # Scrape JS-Tutorials
rake scrape:tutorials:py[num,force] # Scrape PY-Tutorials
rake scrape:tutorials:rb[num,force] # Scrape RB-Tutorials
rake test:curl[apis] # Tests the curl examples of api definitions
rake test:proof # Want some work to do? Run this report and get busy
I use many of these tasks to help with documentation. Here's a rundown of the most useful ones ...
Scrape API Definitions
For example, rake scrape:api_defs is especially useful. As it says, it scrapes the API. But what does that even mean?
When it scrapes the API, it's asking the (full) node what methods are available and what those method signatures are. Then, it saves the scraped information to a .yml file so that jekyll can make pages to browse. For example, rc_api is based on scraped data saved to rc_api.yml.
I don't just scrape the methods, though. Once scraped, I add details. Also, sometimes the scrape is missing some of them method signatures, so I add those too.
You might think this is only useful for catching up to methods that already exist, but it's also possible to use it for upcoming methods. All I need is to point the task at a node running on a future release. Often, new methods show up on api.steemitdev.com, so this will scrape from there:
TEST_NODE=https://api.steemitdev.com bundle exec rake scrape:api_defs
In fact, I could scrape from a testnet just as easily:
TEST_NODE=https://testnet.steemitdev.com bundle exec rake scrape:api_defs
It also works just as well to run a tintoy testnet if I'm looking for bleeding edge new methods:
docker run -d -p 8090:8090 inertia/tintoy:latest
# need to wait for it to fully deploy
sleep 300
TEST_NODE=http://localhost:8090 bundle exec rake scrape:api_defs
And here's the result when I ran it just now:
Definitions for: account_by_key_api, methods: 1
Definitions for: account_history_api, methods: 4
Adding: enum_virtual_ops
Changed: get_account_history
Changed: get_ops_in_block
Changed: get_transaction
4 methods added/updated in account_history_api
Definitions for: block_api, methods: 2
Definitions for: condenser_api, methods: 84
Dropped method: get_account_bandwidth (recommend removal from _data/apidefinitions/condenser_api.yml)
Definitions for: database_api, methods: 52
Changed: find_accounts
Changed: find_comments
Changed: find_escrows
Adding: find_smt_token_emissions
Adding: find_smt_tokens
Changed: find_vesting_delegation_expirations
Changed: find_votes
Changed: find_witnesses
Changed: get_active_witnesses
Changed: get_config
Changed: get_dynamic_global_properties
Adding: get_nai_pool
Changed: get_transaction_hex
Changed: get_witness_schedule
Changed: list_accounts
Changed: list_comments
Changed: list_escrows
Changed: list_limit_orders
Changed: list_owner_histories
Changed: list_savings_withdrawals
Changed: list_sbd_conversion_requests
Adding: list_smt_token_emissions
Adding: list_smt_tokens
Changed: list_vesting_delegation_expirations
Changed: list_vesting_delegations
Changed: list_votes
Changed: list_withdraw_vesting_routes
Changed: list_witness_votes
Changed: list_witnesses
29 methods added/updated in database_api
Definitions for: follow_api, methods: 10
Definitions for: jsonrpc, methods: 2
Definitions for: market_history_api, methods: 7
Definitions for: network_broadcast_api, methods: 2
Definitions for: rc_api, methods: 3
Changed: find_rc_accounts
Changed: get_resource_params
Changed: get_resource_pool
3 methods added/updated in rc_api
This is how I can be ahead of any changes that might roll out. In this example, there seems to be a lot of changes, but most of them are just method signatures with keys in different orders. The most interesting results, to me, are things like Adding: get_nai_pool.
If we focus only on that result, here's what we see newly added to database_api.yml:
- api_method: database_api.get_nai_pool
purpose:
parameter_json: "{}"
expected_response_json: '{"nai_pool":[]}'
What I usually do at this point is search for the new method in the steemd repo. In this case, that leads to database_api.cpp and pull request #3058. I keep working backwards until I find the reason the method was added. Actually, this one is a little tricky. Looks like I need to investigate issue #2886. But I'll do that later.
Test Curl
The rake test:curl[apis] task is like a smoketest for the curl examples in the API Definitions. The rationale is that if we assume these curl examples are correct, then testing them periodically should show us problems in the API and/or node configuration. Conversely, it could also tell is if our documentation is out of date. Either way, running this rake task will give us a hint on how out of date documentation is.
The tests also accept node options, so we can test against pre-release/testnets and get ahead of any changes that might happen before they go to production.
Ops Dump
As mentioned in the rake ops_dump. When we run it, we get:
account_create
account_create_with_delegation
account_update
account_witness_proxy
account_witness_vote
cancel_transfer_from_savings
challenge_authority
change_recovery_account
claim_account
claim_reward_balance
comment
comment_options
convert
create_claimed_account
custom
custom_binary
custom_json
decline_voting_rights
delegate_vesting_shares
delete_comment
escrow_approve
escrow_dispute
escrow_release
escrow_transfer
feed_publish
limit_order_cancel
limit_order_create
limit_order_create2
pow
pow2
price
prove_authority
recover_account
report_over_production
request_account_recovery
reset_account
set_reset_account
set_withdraw_vesting_route
transfer
transfer_from_savings
transfer_to_savings
transfer_to_vesting
vote
withdraw_vesting
witness_set_properties
witness_update
These are the broadcast ops that the devportal currently knows about, not including virtual ops. Broadcast ops are typically more important to app developers than virtual ops because they represent actions that applications can perform, whereas virtual ops are actions that the blockchain performs, typically delayed actions based on timestamps and other internal criteria.
Once we know about a new op, we can add it to the broadcast_ops.yml file which Jekyll uses to generate:
https://inertia186.github.io/devportal/apidefinitions/broadcast-ops
Test Proof
The rake test:proof task has Jekyll build the site, then run basically a lint check on the site. This allows us to find problems with the site. I also use Site Sucker to further validate the results.
Other Tasks
The other tasks, like rake production:deploy and rake production:rollback are just used to build the site for use on production environments or switch back to the previous version. We need to do this externally instead of relying on Github Pages because there are various non-standard libraries required to build the site and Github doesn't like running non-standard libraries.
We also have scrape tasks that will pull tutorials in from other repositories.
Resources
- To get a local copy of the devportal and try these tasks yourself: https://github.com/inertia186/devportal/blob/master/readme.md
Previous Posts of this Series
- More Current Steem Documentation (original fork announcement)
- Documenting Reputation and Database API Definitions
- Full Node Quickstart