A model is a simplified, abstracted representation of an object or system that presents only the information needed by its user. For example, the plastic models of aircraft I built as a kid abstract away everything except the external appearance, a mathematical model of a system shows only those dimensions and relationships useful to the model's users, a control system is a model of the relationships between the stimuli and the response desired by the designer and user of the larger system being controlled (evolution as designer and organism as user in biological analogy). A control system doesn't make a model of a system, to a large degree it is the designers' model of the system it controls.
At the simplest end are one-dimensional models, that we call measurements.
The most complex models are not explicit, they are too complex to be explicitly known, much less communicated; the model of the world that each person carries within his own mind.
Showing posts with label futurism. Show all posts
Showing posts with label futurism. Show all posts
Friday, October 30, 2009
Saturday, October 17, 2009
Risks, Actions, and Benefits
Partially a response and clarification (at least I think it's clearer) to the strategies presented in Alicorn's post on Less Wrong, The Shadow Question. One of the big problems in her discussion of strategies is the conflation of up-front costs with risks.
"First, Do No Harm": When it is as easy to make things worse as better, be damn sure you know what you're doing before you start fixing things.
"Cherry on Top": An invitation to fiddle; small changes are very unlikely to make things worse, and may help.
"Lottery Ticket": She talks about a risk of making things worse, but it looks more like (from her examples and general discussion) that she means is an upfront cost with a chance of significant benefit later.
Insurance: The other headings were hers, but the one she uses here is misleading, as is her discussion. This is related to "Lottery Ticket" in having upfront costs, but in this case it's to prevent an unacceptable risk of harm. It can be something as simple as insuring your house against fire, so you have a temporary place to live and your house gets repaired (or you get a new house if that's easier/cheaper). To actually working to make a risky future less likely (for example, working on Friendly AI).
Another strategy mentioned by Morendil in a comment is "Go for broke" (a less functional version of this would be Russian Roulette), a big risk with the chance of a big reward, like First, Do No Harm, but higher potential risk/payoff matrix.
First, Do No Harm - Use knowledge to avoid as much risk as possible while still seeking the reward
Go for broke - Straightforward acceptance of large risk with large reward
Cherry on Top - Seek benefits at minimal risk
Lottery Ticket - Pay an up-front cost for a small chance at a large benefit
Insurance - Accept an up-front cost to hedge against a risk
Adventure sports isn't a risk management strategy, I mention it here because it feels like there should be a benefit - Seek the thrill of risk, while reducing actual risks, and not getting any benefit except the thrill
If you think of any other generic strategies, please leave a mention in the comments.
As an aside:
As for the title of the original post, I had to Google "Shadow Question". I don't watch television and have never seen an episode of Babylon 5. Given the page I found that describes the show, that was no loss. But the "two questions", "Who are you?" is the Vorlon question. "What do you want?" is the Shadow question. I guess you could call the first one silliness, and the Shadow question practical.
"First, Do No Harm": When it is as easy to make things worse as better, be damn sure you know what you're doing before you start fixing things.
"Cherry on Top": An invitation to fiddle; small changes are very unlikely to make things worse, and may help.
"Lottery Ticket": She talks about a risk of making things worse, but it looks more like (from her examples and general discussion) that she means is an upfront cost with a chance of significant benefit later.
Insurance: The other headings were hers, but the one she uses here is misleading, as is her discussion. This is related to "Lottery Ticket" in having upfront costs, but in this case it's to prevent an unacceptable risk of harm. It can be something as simple as insuring your house against fire, so you have a temporary place to live and your house gets repaired (or you get a new house if that's easier/cheaper). To actually working to make a risky future less likely (for example, working on Friendly AI).
Another strategy mentioned by Morendil in a comment is "Go for broke" (a less functional version of this would be Russian Roulette), a big risk with the chance of a big reward, like First, Do No Harm, but higher potential risk/payoff matrix.
First, Do No Harm - Use knowledge to avoid as much risk as possible while still seeking the reward
Go for broke - Straightforward acceptance of large risk with large reward
Cherry on Top - Seek benefits at minimal risk
Lottery Ticket - Pay an up-front cost for a small chance at a large benefit
Insurance - Accept an up-front cost to hedge against a risk
Adventure sports isn't a risk management strategy, I mention it here because it feels like there should be a benefit - Seek the thrill of risk, while reducing actual risks, and not getting any benefit except the thrill
If you think of any other generic strategies, please leave a mention in the comments.
As an aside:
As for the title of the original post, I had to Google "Shadow Question". I don't watch television and have never seen an episode of Babylon 5. Given the page I found that describes the show, that was no loss. But the "two questions", "Who are you?" is the Vorlon question. "What do you want?" is the Shadow question. I guess you could call the first one silliness, and the Shadow question practical.
Thursday, July 16, 2009
Existential Risk Mitigation
This post is partially a response to lesswrong.com/lw/12h/our_society_lacks_good_selfpreservation_mechanisms
From the Less Wrong post (I replaced the bullets with numbers for ease of reference):
"Well, Bostrom has a paper on existential risks, and he lists the following risks as being "most likely":
1 Deliberate misuse of nanotechnology,
2 Nuclear holocaust,
3 Badly programmed superintelligence,
4 Genetically engineered biological agent,
5 Accidental misuse of nanotechnology (“gray goo”),
6 Physics disasters,
7 Naturally occurring disease,
8 Asteroid or comet impact,
9 Runaway global warming,
10 Resource depletion or ecological destruction,
11 Misguided world government or another static social equilibrium stops technological progress,
12 “Dysgenic” pressures (We might evolve into a less brainy but more fertile species, homo philoprogenitus “lover of many offspring”)
13 Our potential or even our core values are eroded by evolutionary development,
14 Technological arrest,
15 Take-over by a transcending upload,
16 Flawed superintelligence,
17 [Stable] Repressive totalitarian global regime, "
First, there are very few real existential risks.
Of these 3 and 16 are the same problem, and 15 is close enough.
And so are 11 and 14.
9, 10, 12, and 13 are not real problems.
2 is not an existential risk.
11, 14, and 17 are not existential problems in themselves, although they could limit our ability to deal with a real existential problem if one arose.
So that leaves:
1 Deliberate misuse of nanotechnology
3/15/16 Flawed superintelligence
4 Genetically engineered biological agent
5 Accidental misuse of nanotechnology
6 Physics disasters
7 Naturally occurring disease
8 Asteroid or comet impact
6 is not likely and the only way to prevent it is deliberately impose 11/14, which while not an existential risk itself will increase the difficulty in handling an existential (or other) danger that may eventually occur.
7 and 8 are so unlikely within any given time span that they are not worth worrying about until the other dangers can be handled.
I used to think 1 was most likely and 5 next, but Eliezer Yudkowsky's writings have convinced me that unfriendly AI (3/15/16) is a nearer term risk, even if not necessarily a worse one.
Libertarianism is the best available self-preservation mechanism. I am using libertarianism in a general sense of freedom from government interference. It is the social and memetic equivalent of genetic behavioral dispersion; that members of many species behave slightly differently which reduces the likelihood of a large percentage falling to the same cause. The only possible defense against the real risks is to have many people researching them from many different directions - the biggest danger with any of these only occur if someone has a substantial lead in the development/implementation of the technologies involved.
From the Less Wrong post (I replaced the bullets with numbers for ease of reference):
"Well, Bostrom has a paper on existential risks, and he lists the following risks as being "most likely":
1 Deliberate misuse of nanotechnology,
2 Nuclear holocaust,
3 Badly programmed superintelligence,
4 Genetically engineered biological agent,
5 Accidental misuse of nanotechnology (“gray goo”),
6 Physics disasters,
7 Naturally occurring disease,
8 Asteroid or comet impact,
9 Runaway global warming,
10 Resource depletion or ecological destruction,
11 Misguided world government or another static social equilibrium stops technological progress,
12 “Dysgenic” pressures (We might evolve into a less brainy but more fertile species, homo philoprogenitus “lover of many offspring”)
13 Our potential or even our core values are eroded by evolutionary development,
14 Technological arrest,
15 Take-over by a transcending upload,
16 Flawed superintelligence,
17 [Stable] Repressive totalitarian global regime, "
First, there are very few real existential risks.
Of these 3 and 16 are the same problem, and 15 is close enough.
And so are 11 and 14.
9, 10, 12, and 13 are not real problems.
2 is not an existential risk.
11, 14, and 17 are not existential problems in themselves, although they could limit our ability to deal with a real existential problem if one arose.
So that leaves:
1 Deliberate misuse of nanotechnology
3/15/16 Flawed superintelligence
4 Genetically engineered biological agent
5 Accidental misuse of nanotechnology
6 Physics disasters
7 Naturally occurring disease
8 Asteroid or comet impact
6 is not likely and the only way to prevent it is deliberately impose 11/14, which while not an existential risk itself will increase the difficulty in handling an existential (or other) danger that may eventually occur.
7 and 8 are so unlikely within any given time span that they are not worth worrying about until the other dangers can be handled.
I used to think 1 was most likely and 5 next, but Eliezer Yudkowsky's writings have convinced me that unfriendly AI (3/15/16) is a nearer term risk, even if not necessarily a worse one.
Libertarianism is the best available self-preservation mechanism. I am using libertarianism in a general sense of freedom from government interference. It is the social and memetic equivalent of genetic behavioral dispersion; that members of many species behave slightly differently which reduces the likelihood of a large percentage falling to the same cause. The only possible defense against the real risks is to have many people researching them from many different directions - the biggest danger with any of these only occur if someone has a substantial lead in the development/implementation of the technologies involved.
Subscribe to:
Posts (Atom)