Existential Risk Mitigation

This post is partially a response to lesswrong.com/lw/12h/our_society_lacks_good_selfpreservation_mechanisms


From the Less Wrong post (I replaced the bullets with numbers for ease of reference):

"Well, Bostrom has a paper on existential risks, and he lists the following risks as being "most likely":

1 Deliberate misuse of nanotechnology,
2 Nuclear holocaust,
3 Badly programmed superintelligence,
4 Genetically engineered biological agent,
5 Accidental misuse of nanotechnology (“gray goo”),
6 Physics disasters,
7 Naturally occurring disease,
8 Asteroid or comet impact,
9 Runaway global warming,
10 Resource depletion or ecological destruction,
11 Misguided world government or another static social equilibrium stops technological progress,
12 “Dysgenic” pressures (We might evolve into a less brainy but more fertile species, homo philoprogenitus “lover of many offspring”)
13 Our potential or even our core values are eroded by evolutionary development,
14 Technological arrest,
15 Take-over by a transcending upload,
16 Flawed superintelligence,
17 [Stable] Repressive totalitarian global regime, "

First, there are very few real existential risks.

Of these 3 and 16 are the same problem, and 15 is close enough.
And so are 11 and 14.
9, 10, 12, and 13 are not real problems.
2 is not an existential risk.
11, 14, and 17 are not existential problems in themselves, although they could limit our ability to deal with a real existential problem if one arose.

So that leaves:

1 Deliberate misuse of nanotechnology
3/15/16 Flawed superintelligence
4 Genetically engineered biological agent
5 Accidental misuse of nanotechnology
6 Physics disasters
7 Naturally occurring disease
8 Asteroid or comet impact

6 is not likely and the only way to prevent it is deliberately impose 11/14, which while not an existential risk itself will increase the difficulty in handling an existential (or other) danger that may eventually occur.

7 and 8 are so unlikely within any given time span that they are not worth worrying about until the other dangers can be handled.

I used to think 1 was most likely and 5 next, but Eliezer Yudkowsky's writings have convinced me that unfriendly AI (3/15/16) is a nearer term risk, even if not necessarily a worse one.

Libertarianism is the best available self-preservation mechanism. I am using libertarianism in a general sense of freedom from government interference. It is the social and memetic equivalent of genetic behavioral dispersion; that members of many species behave slightly differently which reduces the likelihood of a large percentage falling to the same cause. The only possible defense against the real risks is to have many people researching them from many different directions - the biggest danger with any of these only occur if someone has a substantial lead in the development/implementation of the technologies involved.

Some Notes on Responsibility

Slightly revised version of some comments I left on lesswrong.com - Does Blind Review Slow Down Science?

First I should state that I disagree with anonymous review for the same reasons that I disagree with an unaccountable judiciary - the negative effects on responsibility.

However, there are several problems with the theory in this essay - the most important being that the editors know who the writer or researcher is and can decide to go ahead and publish on that score no matter what the reviewers say. The editors have a strong incentive to advance novel but true theories in that it will advance the reputation of the journal.

About the unaccountable judiciary, you might check out this book by Max Boot, Out Of Order: Arrogance, Corruption, And Incompetence On The Bench, a large proportion of the problems he wrote about arose from judges not being personally responsible for their actions on the bench.

Also, more generally, I am a libertarian largely because I believe that everyone is totally and completely responsible for their own actions. Even if someone is holding a gun to your head, you decide what you do in response (and are responsible for letting yourself get in that position). Or if you are drunk or drugged, you are responsible for putting yourself in that position and therefore for what you do while that way.

By "responsible", I mean that people should bear some part of the forseeable costs of their actions. I say "some part" because the actions of others also influence costs, and stress "foreseeable" because in any complex system things interact to such an extent that only very direct results can actually be attributed reliably to any one party. Most attributions of "fault" in complex systems is scapegoating or motivated by interpersonal status games.

Trial and Error Problem Solving

The most basic requirement of successful use of trial and error is all too often forgotten - you must keep track of what you have tried.

Systemic trial and error is based on a mutually exclusive and exhaustive listing of all possible solutions. In many real world problems this isn't possible, but it works in some areas such as mathematics and some areas of engineering. Even just attempting an exhaustive listing of solutions can help you find possible solutions that you may have overlooked otherwise, even if a truly exhausitve listing is not possible.

Alarm Clocks, Scheduling, and Deadlines

It's probably better not to set goals in the first place, if you are not going to keep them, than to get in the habit of ignoring them. I make it a point now not to set my alarm clock if I don't have to be up by particular time, and if I do set it to always get up.

Similarly, I try not to set a fixed schedule for things I don't need to do at a particular time. I make a list and work on something from the list (unless something gets near a deadline). I found that having the list also helps me to avoid spinning my wheels thinking about what to work on (provided your list isn't too long, then you have to decide what to work on).

Making arbitrary "deadlines" for yourself, then ignoring them, increases rather than decreases your procrastination.

General Method of Problem Solving

1. Become conscious of problem
Identify opportunities to improve situation.
Identify opportunities for innovation.

2. Recognize and define problem.

3. Define the goal.

4. Search for and develop alternative solutions.
Explore possible strategies for achieving goals.
Formulate hypotheses.

5. Select alternative solution or strategies. Anticipate the precise outcome. Choose the hypothesis to test.

6. Implement solution or strategy. Act on it. Commit through action. Test the hypothesis (do the experiment).

7. Gain feedback and compare to anticipated outcome. Refine the solution.


Trying too hard to avoid complications often creates them.

When in total ignorance, try anything, at the least you will learn something that doesn't work. And watching how it fails should teach you a lot more than that. Seeing to what extent it looked like it could work before it finally failed should reveal interesting data also.

Edited to Add References: I was in a hurry yesterday when I posted the above and didn't have time to lokk these up in Amazon.

The greatest source for the above is The Ideal Problem Solver: A Guide to Improving Thinking, Learning, and Creativity by John D Bransford and Barry S Stein. The IDEAL in the title is an acronym for a five-step process, similar to the 7 steps I listed above.

Next was Wayne A Wickelgren's How to Solve Mathematical Problems, the most complete view of problem solving techniques and heuristics I have seen. Some of the techniques don't work well on less well defined types of problems, but most do.

Polya's How to Solve It: A New Aspect of Mathematical Method (Princeton Science Library) had a significant impact when I read it, but that was more than 15 years ago. I strongly recommend this to anyone interested in math or general problem solving, before reading Wickelgren's book.

Two more books that made a significant impression on me at the time I read them, but that I didn't actually refer to for this are Thinking Better by David Lewis and James Greene and an earlier edition of Problem Solving & Comprehension.

I am currently reading Thinking and Deciding by Jonathan Baron. It is a more difficult read, and more psychologically oriented than the others. The author's claim that effectively all thinking, deciding, problem solving, creativity are a combination of search and selection is interesting and looks very useful.

Law and Outlawry

Justification of Law (Mala in se)

Government law is "justified", really defended in the psychological sense, by claiming to be needed to enforce prohibitions of universally recognized wrongs (mala in se), such as robbery and murder.

Perversion of Law (Mala prohibita)

Government perverts the law (mala in se) by making laws (mala prohibita) which mainly favor specific groups - mainly to increase its own power over its subjects/citizens/serfs.

Obedience to Law as Prudential Rule

Breaking the law, or even appearing to do so, increases the risks to your values (especially life and freedom) from government agents and their hordes of unpaid informers that infest the country. Given the errors and arbitrary enforcement (and willing dishonety), everyone who is not themselves members of the government is at some risk of attack by government agents.

Living Outside the Law

Given the various risks and dangers, on both sides of the divide, your values may be more effectively served by living outside the law. I don't think so for various reasons, but it is arguable enough that I decided to include a section discussing it.

Historically, the outlaw was not necessarily a criminal - but one who for whatever reason lived outside of the limits and protections of the law. It was often used as a punishment, sort of an extension of shunning and ostracism, sometimes to an explicit withdrawl of the law's protection - where anyone who wanted to, and could manage it, could kill the outlaw. But it was sometimes used by those who simply wanted to be left alone, given human nature any outlaws usually became the scapegoats for anything that went wrong in the area.

The growth of government power and the use of mala prohibita is likely closely related to the decline of voluntary outlawry.

The only benefit today in living outside the law is reduction in tax burden. Most other possible benefits I have heard of you can actually get legally with a little thought and care. And even the benefit of tax avoidance is questionable since it is harder to earn income outside the law, than it is without the need to evade detection. If discovered, or even suspected, it would be dangerous for the outlaw/evader and with modern computer databases and communications it would be likely to be noticed, and ever more likely to be noticed in the future.

Trying to live outside the law is not generally worth it, but may be worth keeping in mind. The only way it would be cost-effective would be if you were interested in
an extremely low, subsistence-level life where you would need almost no income. In that case you could do handyman work and day labor for cash. The biggest problem with that is getting older. Long before retirement age, if you are doing physical labor, you are going to start aching, eventually more or less continuously. I
don't recommend it.

Are Best Practices the New Mediocrity

Good point - if everyone is doing something it's mediocre, even if it's called "best practices". Probably, the use of "best practices" should be replaced with "standard practices", which is an almost as common synonym, and is more accurate.

The Road to Mediocrity is Paved with Best Practices