Community is a sphere of interest, which includes artificial intelligence (AI) and futurology investigation of existential risks (X-risk), in recent years more, and more shows considerable interest in blockchain and crypto economical systems. Ralph Merkle, inventor of the now famous cryptographic technology, which underpins Ethereum’s light client protocol, has expressed interest in DAO governance. Skype co-founder Jaan Tallinn proposed researching blockchain technology as a way to create mechanisms to solve global coordination problems. Prediction market advocates, who have long understood the potential of prediction markets as governance mechanisms, are now looking at Augur. Is there anything interesting here? Is this simply a situation of computer geeks who were previously attracted to computer-geek-friendly topic A now also being attracted to a completely unrelated but also computer-geek-friendly topic B, or is there an actual connection?
I am satisfied that there is a connection. Both the crypto economics research community and the AI safety / computer-assisted control/ X-risk from different perspectives approach one and the same problem: is it possible to regulate intelligent and complex systems with unpredictable and spontaneous emergent properties, with simple and "dumb" system with predetermined and constant parameters?
In the context of AI research, there is a problem of function definition, that would guide the behavior of a super intelligent agent and however keep out of executes an action, formally satisfies the predetermined behavior, but does not satisfy the intent (Edge Instantiation –extreme solution). For example, AI receive instruction to cure cancer, it may end up reasoning that the most reliable way to do that is to simply kill everyone first. If you tried to plug that hole, it may decide to simply permanently cryogenically freeze all humans without killing them. And so forth. (Recall the late Lenin: " The form is correct, the essence - mockery").
In the context of Ralph Merkle’s DAO democracy, the problem is that of determining an objective function that is correlated with social and technological progress and generally things that people expect, but in no way correlated with existential risks, and is easily measurable enough that its measurement would not itself become a source of political battles.
Meanwhile, in the context of crypto economics, the problems are surprisingly similar. The core problem of consensus asks: how to incentivize validators in continuation of supporting and maintaining general records by dint of simple algorithm that is “set in stone”, while the validators themselves are highly complex economic system, interact against each other in a free ways, and often non-concur or cross purposes. The DAO incident became consequence of divergence of software developers’ complex intent, having a specific use in mind for the splitting function, and the de-facto result of the software implementation. Augur work at supplement of consensus problem to real-world facts. Maker is trying to create a decentralized governance algorithm for a platform that intends to provide an asset with the decentralization of cryptocurrency and the reliability of fiat currency. In all of these cases, the algorithms are “dumb”, and yet the agents that they have to control are “smart”.AI safety: agents with IQ 150 trying to control agents with IQ 6000, whereas crypto economics is about agents with IQ 5 trying to control agents with IQ 15. Certainly, problems are different, but where the similarities are not to be fob off.
These are problems, which both communities have already been separately considering for many years and have in some cases accumulated considerable experience. The solutions are being used heuristic partial solutions and mitigation strategies. In creating DAOs, some developers are moving toward a hybrid approach that has a set of curators with some control over the DAO’s assets, but assigns those curators only limited powers. Their capacity is sufficient to prevent an attack on the DAO, but not enough to carry out its own full-scale attack - this approach is similar to research in the field of AI safe interruption.
On the futarchy side, we see interest in the use of interest rates as an objective function, combined futarhii and quadratic voting through the voluntary freezing of funds, as the control algorithm (POS). This may also include various hybrid forms that convey futarhii power sufficient to prevent attacks "majority collusion” way, inaccessible democracy, and at the same time leave the power of the voting process. Such innovations, at least, deserve discussion in the group, which considers futarchy as a construction of DAO democracy.
Other previously unappreciated solution - the use of control mechanisms explicitly delaying the development of undesirable processes: proposed DAO hard fork that may rescue the contained funds from the DAO Black is only possible precisely because the DAO included a set of rules that required every action to have a long delay time. Another new direction - formal verification. It is using computer programs that verify other computer programs. The goal – to make sure that the results of the programs verifiable performance satisfy the purposes for which they created.
Formally proving “honesty” in the general case is impossible, due to the complexity of value problem (in general that human values have a high Kolmogorov complexity). However, we use different ways to reduce risk. For example, we could formally prove that a certain kind of action cannot be taken in less than 7 days, or that a certain kind of action cannot be taken for 48 hours, if the curators of the DAO will vote a certain way. In the community of AI, researchers are preservatives used to prevent various errors in the award function. These errors can lead to a completely unexpected behavior of the system due to the extremely high cost of the complexity of the AI. Of course, the study of formal verification techniques conducted by various communities for many years, but now we consider their application in additional environment
Another concept, popular in the field of AI security, which can be extremely useful in the construction of economic systems based on the DAO - super rational decision theories. This ways to overcome prisoner’s dilemma situations by committing to run source code that treats agents, which also commit to run that source code more favorably. One example of a move available to open-source agents that is not available to “black box” agents is the “values handshake” described in a short story by Scott Alexander: two mutually suspicious entities joint proceed to target which is the average of the two goals that they previously had. Previously, such concepts were largely science fiction, but now futarchy DAOs allows implementing them in practice. More generally, a DAO may well be a highly effective means for a social institution to strongly commit to “running source code” that has particular properties.
The DAO - only the first of many that will start in this and next years, and there is no doubt that all subsequent draw a lesson of the first one. They will bring a lot of new: security policy, control algorithms, the curators of the structure, and the delay system with safety mechanisms, and formally verified guarantee code. All this will help sustain crypto economic storm.
A few final words: the biggest lesson for crypto-community is the very notion of decentralization: the different groups duplicating work performance, so minimizing the likelihood that an error of one system will pass unnoticed by other systems. Crypto ecosystem is still evolving and will become the current experiment at the forefront of software development, computer science, game theory and philosophy. Regardless of whether it will be the mainstream of social applications in its current form, or after a few iterations, the results will be available to anyone, able to learn and see.
Author: Vitalik Buterin
Source:Медиум