Scribd Logo
Hochladen

Willkommen bei Scribd!

  • AI Safety Levels

    Hochgeladen von

    Turchin Alexei
    104 Ansichten1 Seite
    One of the main principles of engineering safety is multilevel defence. When a nuclear bomb accidentally fell from the sky in the US, 3 of 4 defence levels failed. The last one prevented the nuclear explosion: https://en.wikipedia.org/wiki/1961_Goldsboro_B-52_crash Multilevel defence is used a lot in the nuclear industry and includes different systems of passive and active safety, starting from the use of delayed neutrons for the reaction activation and up to control rods, containment building and exclusion zones. Here, I present a look at the AI safety from the point of view of multilevel defence. This is mainly based on two of my yet unpublished articles: “Global and local solutions to AI safety” and “Catching treacherous turn: multilevel AI containment system”.  The special property of the multilevel defence, in the case of AI, is that the biggest defence comes from only the first level, which is AI alignment. Other levels have progressively smaller chances to provide any protection, as the power of self-improving AI will grow after it will break of each next level. So we may ignore all levels after AI alignment, but, oh Houston, we have a problem: based on the current speed of AI development, it seems that powerful and dangerous AI could appear within several years, but AI safety theory needs several decades to be created.  The map is intended to demonstrate a general classification principle of the defence levels in AI safety, but not to list all known ideas on the topic. I marked in “yellow” boxes, which are part of the plan of MIRI according to my understanding.  I also add my personal probability estimates as to whether each level will work (under the condition that AI risks are the only global risk, and previous levels have failed). The principles of the construction of the map are similar to my “plan of x-risks prevention” map and my “immortality map”, which are also based around the idea of the multilevel defence.

    Copyright

    © © All Rights Reserved

    Verfügbare Formate

    PDF, TXT oder online auf Scribd lesen

    Stufen Sie dieses Dokument als nützlich ein?

    Sind diese Inhalte unangemessen?

    Dieses Dokument melden
    One of the main principles of engineering safety is multilevel defence. When a nuclear bomb accidentally fell from the sky in the US, 3 of 4 defence levels failed. The last one prevented the nuclear explosion: https://en.wikipedia.org/wiki/1961_Goldsboro_B-52_crash Multilevel defence is used a lot in the nuclear industry and includes different systems of passive and active safety, starting from the use of delayed neutrons for the reaction activation and up to control rods, containment building and exclusion zones. Here, I present a look at the AI safety from the point of view of multilevel defence. This is mainly based on two of my yet unpublished articles: “Global and local solutions to AI safety” and “Catching treacherous turn: multilevel AI containment system”.  The special property of the multilevel defence, in the case of AI, is that the biggest defence comes from only the first level, which is AI alignment. Other levels have progressively smaller chances to provide any protection, as the power of self-improving AI will grow after it will break of each next level. So we may ignore all levels after AI alignment, but, oh Houston, we have a problem: based on the current speed of AI development, it seems that powerful and dangerous AI could appear within several years, but AI safety theory needs several decades to be created.  The map is intended to demonstrate a general classification principle of the defence levels in AI safety, but not to list all known ideas on the topic. I marked in “yellow” boxes, which are part of the plan of MIRI according to my understanding.  I also add my personal probability estimates as to whether each level will work (under the condition that AI risks are the only global risk, and previous levels have failed). The principles of the construction of the map are similar to my “plan of x-risks prevention” map and my “immortality map”, which are also based around the idea of the multilevel defence.

    Copyright:

    © All Rights Reserved
    Als PDF, TXT herunterladen oder online auf Scribd lesen
    Markieren Sie unangemessene Inhalte
    104 Ansichten1 Seite

    AI Safety Levels

    Hochgeladen von

    Turchin Alexei
    One of the main principles of engineering safety is multilevel defence. When a nuclear bomb accidentally fell from the sky in the US, 3 of 4 defence levels failed. The last one prevented the nuclear explosion: https://en.wikipedia.org/wiki/1961_Goldsboro_B-52_crash Multilevel defence is used a lot in the nuclear industry and includes different systems of passive and active safety, starting from the use of delayed neutrons for the reaction activation and up to control rods, containment building and exclusion zones. Here, I present a look at the AI safety from the point of view of multilevel defence. This is mainly based on two of my yet unpublished articles: “Global and local solutions to AI safety” and “Catching treacherous turn: multilevel AI containment system”.  The special property of the multilevel defence, in the case of AI, is that the biggest defence comes from only the first level, which is AI alignment. Other levels have progressively smaller chances to provide any protection, as the power of self-improving AI will grow after it will break of each next level. So we may ignore all levels after AI alignment, but, oh Houston, we have a problem: based on the current speed of AI development, it seems that powerful and dangerous AI could appear within several years, but AI safety theory needs several decades to be created.  The map is intended to demonstrate a general classification principle of the defence levels in AI safety, but not to list all known ideas on the topic. I marked in “yellow” boxes, which are part of the plan of MIRI according to my understanding.  I also add my personal probability estimates as to whether each level will work (under the condition that AI risks are the only global risk, and previous levels have failed). The principles of the construction of the map are similar to my “plan of x-risks prevention” map and my “immortality map”, which are also based around the idea of the multilevel defence.

    Copyright:

    © All Rights Reserved
    Als PDF, TXT herunterladen oder online auf Scribd lesen
    Markieren Sie unangemessene Inhalte
    von 1

    Levels of defense in AI safety

    Levels Global solutions Local solutions


    Funding of Attracting Creation of AI safety
    0 Preparation the AI safety guidlines and
    the AI safety minds
    reserch theory mindset

    Prelimnary
    measures Prevention
    Destruction Promotion of Slowing AI
    of the of AI labs
    Restriction
    AI safety idea
    by law development
    Strong AI everywhere and theory
    creation

    Limiting AI Creating Preventing self- Messy AIs and


    scale and Manipulating
    many AIs for improvement uploads
    impact AI arms race
    balansing? of an AI

    Solving AI EthicsNet dataset Correct decesion Provably safe Human Human


    1 alignment competition theory AI design emulation self-improving
    problem
    Safe
    AI Installing CEV AI Vague values
    correct constitution
    design goals

    Solving Taking over Creating AI First AI creators will


    other AI the world police implement safe AI
    problem theory

    Safely interruptable Red button and


    2 Control agents; Corrigibility other internal
    Circuit-breakers
    systems constrains
    Catching
    treacherous
    turn Testing
    Testing in Testing in Narrow AI
    simulation small tasks to control AI

    3 Information Disconection from Phisically remote


    limiting interent place
    AI boxing

    Instruments
    4 to affect Messaging to Utility monsters
    mature AI future Young AI and landmines
    Stopping Created by Alexey Turchin, 2017, CC4.0
    See also other my maps on the topic:
    AI failures modes
    AI AI safety solution
    and articles
    Hopes to
    in the wild survive AI Humans Humans as AI will
    Message to any future AI
    Catching treacherous turn MIRI strategy (as
    in simulation slaves abandon Earth Global and local solutions of AI safety I understand it)
    Human upload based AI Nanny marked in yellow

    Time

    Das könnte Ihnen auch gefallen