On Numerai, data scientists use machine learning to make predictions that power Numerai’s hedge fund. Numeraire is the cryptographic token of Numerai that can be used to stake predictions and earn more money in Numerai's tournament. I recently stumbled across this and found it utterly fascinating.
I'll break it down by the basic rules,
Datasets
(numerai_training_data.csv [sign up on Numerai to get access to this dataset]) You would use this dataset to train your machine learning algorithm. The target column is the binary class you are trying to predict. The id, era and data_type columns are meta data, not features. All other columns are features.
numerai_tournament_data.csv Use your model to generate probability estimates on this dataset to submit to Numerai. This dataset contains validation data with targets, test data without targets, and live data without targets.
Uploading predictions
The format of your prediction upload should be a CSV file with two columns: id and probability just like the example_predictions.csv file in the dataset. The probability column is the probability estimated by your model of the observation being of class 1.
Leaderboard
The leaderboard displayed is based only on validation data.
To be on the leaderboard, models are required to have concordance, originality, and consistency.
Concordance is a measure of whether predictions on the validation set, test set, and live set appear to be generated by the same model. A data scientist who submits perfect answers on the validation set is unlikely to achieve concordance.
Originality is a measure of whether a set of predictions is uncorrelated with predictions already submitted. Numerai wants to encourage new models over duplicate submissions.
Consistency measures the percentage of eras in which a model achieves a logloss < -ln(0.5). Numerai wants models that work well consistently across eras. Only models with consistency above 75% are considered consistent.
Earnings
Only models with concordance, originality, and consistency control the capital in Numerai’s hedge fund. As such, these are the only models eligible to earn bitcoin or Numeraire.
Earnings are paid out based on live performance. It takes one month for the market to decide the outcome of predictions. Data scientists are paid based on the rank of their realized logloss over that month according to this schedule. For example, if you came first in this week's tournament, you would earn $400 and 1600 Numeraire one month after the start of the tournament. Only models that achieve a live logloss < -ln(0.5) are eligible for payout. The top 100 ranks ordered by live performance logloss will be paid each round. You can participate in every week's tournament.
Over time, data scientists who regularly achieve concordance, originality, consistency and strong live logloss will also earn bonuses as their reputation grows.
Deposit NMR
If you are granted NMR from Numerai, it appears in your user account automatically. If you have NMR in an ethereum address, you may deposit it to your account by sending it your account address using your favorite etherum client software. Your account address can be found by clicking "Balances".
Withdraw NMR
You may withdraw your NMR to another address. This address may be another web account or a personal ethereum address. In either case, click "Balances", and enter the ethereum address and the amount you wish to withdraw. Be careful! This transaction will be recorded permanently on the ethereum blockchain, and if the address is incorrect, your NMR may be lost forever.
Stake NMR
All submissions are eligible for the regular tournament. When you've made a submission that you like, you may place a stake on it to make it eligible for the "staked" tournament. To do so, click "Stake Numeraire", and enter the amount of NMR you wish to stake, as well as your confidence level. The maximum payout (paid in USD) you may receive is the amount of NMR you have staked divided by your confidence level. Stakes have the following restrictions:
You may only stake on yourself.
You may only place a stake if your model is original, concordant, and has consistency >= 75%.
Once you have made a stake, you will not be able to submit any more predictions in this round.
You may only place a stake during the one week the round is open.
You may not stake more NMR than your available balance. Importantly, you may not stake NMR that is already staked on another round, so if you wish to make equal stakes each week, you should stake approximately 1/4 of the amount of NMR you wish to use.
If you have made a stake, you may increase either the amount or the confidence level of your stake. To do so, simply make another stake. Note that you should only put the amount of additional NMR you wish to stake, not the total of your entire stake.
You may not decrease either the amount or the confidence of your stake.
Confidence ties will be broken by who staked first.
When the round resolves, approximately four weeks after it begins, all stakes will be resolved by the auction mechanism described in the whitepaper.
Stake NMR With Custom Ethereum Address
Some advanced users may want to stake from their own ethereum address rather than through Numerai's website. These stakes are the same as stakes made on Numerai's website, it's just another way to make stakes. To do this, we assume you are familiar with sending transactions on the ethereum network, that you have NMR in a personal ethereum account, and that you have enough ether to cover the gas costs of making transactions. Instructions for interacting with the contract are described in the contract's README. Keep the following in mind:
You must add the custom address to your Numerai account before you make a stake from it. You may do this under the "Balances" tab. If you do not do this, your stake will be invalid.
You may add the same custom address to multiple accounts and make separate stakes on each of those accounts.
You must keep the address in your list of custom addresses until the stake is resolved.
Each user may have stake from only one address in a given round. If you make a stake from multiple addresses, the stake from the first address is valid and the others are not.
When staking, you must supply a value, confidence, and a tag. That tag is your username, as a bytes32. Most ethereum clients will accept your username as a string. If you use a tag that does not match a username, or if your account's submission is not original, concordant, or have consistency >= 75%a, then the stake is invalid.
Contact us if you make a stake from a custom address and it does not show up on the leaderboard within an hour.
If your stake is determined to be invalid for any of the above reasons, it will be returned to you when the round resolves four weeks later.
Anonymity and Ownership
You retain all intellectual property rights to your model. You never have to tell anyone how you built it and you never have to tell who you are. You may create up to three accounts to try new models; we limit this to three to prevent spamming the leaderboard. Spammers may be banned or penalized.
Tips
Numerai’s dataset is unusual. It is not a simple time series. It is not a simple binary classification problem. The signal is very low, and it’s transformed with structure-preserving encryption.
The test set, validation set and live set should be treated as hold out sets. We recommend you do not train on the validation data even though you have the targets. If you create features using unsupervised learning be careful not to include the validation set, test set, or live set. Be careful of unsupervised learning generally.
For cross-validation, it’s better to hold out a random sample of eras rather than a random sample rows. Using a random sample of rows tends to over fit.
Cybernetics
If you have a great model, it means you have discovered pernicious pricing errors that are harming the global economy. We can correct these prices together, and move capital to where it belongs. Work together on Slack and on our forums. Our only true adversaries are the other hedge funds.
Numerai's official website: https://numer.ai/
If you liked this article follow me at 
=== Copyright/Copyleft ===
Notwithstanding the provisions of sections 106 and 106A, the fair use of a copyrighted work for purposes such as criticism, comment, review and news reporting is not an infringement of copyright. We are making such material available for the purposes of criticism, comment, review and news reporting which constitute the 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. ALL footage/images used is either done under the express permission of the original owner, or is public domain and falls under rules of Fair Use. (This is a fair use disclaimer that you can use too.)