2024先進(jìn)人工智能安全性評(píng)估國(guó)際科學(xué)報(bào)告中期報(bào)告+（英文版）-132正式版

InternationalScientificReportontheSafetyofAdvancedAIINTERIMREPORTMay2024InternationalScientificReportontheSafetyofAdvancedAI:InterimReportContributorsCHAIRProf.YoshuaBengio,UniversitédeMontréal/Mila-QuebecAIInstituteEXPERTADVISORYPANELProf.BronwynFox,TheCommonwealthScientificandIndustrialResearchOrganisation(CSIRO)(Australia)AndréCarlosPoncedeLeonFerreiradeCarvalho,InstituteofMathematicsandComputerSciences,UniversityofS?oPaulo(Brazil)Dr.MonaNemer,ChiefScienceAdvisorofCanada(Canada)RaquelPezoaRivera,FedericoSantaMaríaTechnicalUniversity(Chile)Dr.YiZeng,InstituteofAutomation,ChineseAcademyofSciences(China)JuhaHeikkil?,DGConnect(EuropeanUnion)GuillaumeAvrin,GeneralDirectorateofEnterprises(France)Prof.AntonioKrüger,GermanResearchCenterforArtificialIntelligence(Germany)Prof.BalaramanRavindran,IndianInstituteofTechnology,Madras(India)Prof.HammamRiza,KORIKA(Indonesia)Dr.CiaránSeoighe,ScienceFoundationIreland(Ireland)Dr.ZivKatzir,IsraelInnovationAuthority(Israel)Dr.AndreaMonti,UniversityofChieti-Pescara(Italy)Dr.HiroakiKitano,SonyGroup(Japan)[Interim]MaryKerema,MinistryofInformationCommunicationsTechnologyandDigitalEconomy(Kenya)Dr.JoséRamónLópezPortillo,QElement(Mexico)

Prof.HaroonSheikh,Netherlands’ScientificCouncilforGovernmentPolicy(Netherlands)Dr.GillJolly,MinistryofBusiness,InnovationandEmployment(NewZealand)Dr.OlubunmiAjala,InnovationandDigitalEconomy(Nigeria)DominicLigot,CirroLytix(Philippines)Prof.KyoungMuLee,DepartmentofElectricalandComputerEngineering,SeoulNationalUniversity(RepublicofKorea)AhmetHalitHatip,TurkishMinistryofIndustryandTechnology(RepublicofTurkey)CrystalRugege,NationalCenterforAIandInnovationPolicy(Rwanda)Dr.FahadAlbalawi,SaudiAuthorityforDataandArtificialIntelligence(KingdomofSaudiArabia)DeniseWong,DataInnovationandProtectionGroup,InfocommMediaDevelopmentAuthority(IMDA)(Singapore)Dr.NuriaOliver,ELLISAlicante(Spain)Dr.ChristianBusch,FederalDepartmentofEconomicAffairs,EducationandResearch(Switzerland)OleksiiMolchanovskyi,ExpertCommitteeontheDevelopmentofArtificialintelligenceinUkraine(Ukraine)MarwanAlserkal,MinistryofCabinetAffairs,PrimeMinister’sOffice(UnitedArabEmirates)SaifM.Khan,U.S.DepartmentofCommerce(UnitedStates)DameAngelaMcLean,GovernmentChiefScientificAdviser(UnitedKingdom)AmandeepGill,UNTechEnvoy(UnitedNations)2InternationalScientificReportontheSafetyofAdvancedAI:InterimReportSCIENTIFICLEADS?renMindermann,Mila-QuebecAIInstituteWRITINGGROUPDanielPrivitera(leadwriter),KIRACenterShayneLongpre,MassachusettsInstituteofTamayBesiroglu,EpochAITechnologyRishiBommasani,StanfordUniversityVasiliosMavroudis,AlanTuringInstituteStephenCasper,MassachusettsInstituteofMantasMazeika,UniversityofIllinoisatTechnologyUrbana-ChampaignYejinChoi,UniversityofWashington/A12KwanYeeNg,ConcordiaAIDanielleGoldfarb,Mila-QuebecAIInstituteChinasaT.Okolo,Ph.D,TheBrookingsHodaHeidari,CarnegieMellonUniversityLeilaInstitutionKhalatbari,HongKongUniversityofScienceDeborahRaji,MozillaandTechnologyTheodoraSkeadas,HumaneIntelligenceFlorianTramèr,ETHZürichSENIORADVISERSJohnA.McDermidOBEFREng,UniversityofBayoAdekanmbi,DataScienceNigeriaPaulChristiano,contributedasaSeniorYorkAdviserpriortotakinguphisroleattheUSAIArvindNarayanan,PrincetonUniversitySafetyInstituteAlondraNelson,InstituteforAdvancedStudyDavidDalrymple,AdvancedResearch+AliceOh,KAISTSchoolofComputingInventionAgency(ARIA)GopalRamchurn,RAIUK/UKRITASThomasG.Dietterich,OregonStateHub/UniversityofSouthamptonUniversityEdwardFelten,PrincetonUniversityStuartRussell,UniversityofCalifornia,PascaleFung,HongKongUniversityofBerkeleyScienceandTechnology,contributedasaMarietjeSchaake,StanfordUniversitySeniorAdviserpriortotakingupherroleatDawnSong,UniversityofCalifornia,BerkeleyMetaPierre-OlivierGourinchas,InternationalAlvaroSoto,PontificiaUniversidadCatólicaMonetaryFund(IMF)deChileNickJenningsCBFREngFRS,UniversityofLeeTiedrich,DukeUniversityLoughboroughGa?lVaroquaux,TheNationalInstituteforAndreasKrause,ETHZurichResearchinDigitalScienceandTechnologyPercyLiang,StanfordUniversity(Inria)TeresaLudermir,FederalUniversityofAndrewYao,InstituteforInterdisciplinaryPernambucoInformationSciences,TsinghuaUniversityVidushiMarda,REALMLYa-QinZhang,TsinghuaUniversityHelenMargettsOBEFBA,UniversityofOxford/AlanTuringInstituteSECRETARIATUKGovernmentSecretariathostedbytheAISafetyInstituteBenjaminPrud’homme,Mila-QuebecAIInstituteACKNOWLEDGEMENTSTheSecretariatappreciatethehelpfulsupport,comments,andfeedbackfromthefollowingUK-basedorganisations:AdaLovelaceInstitute,TheAlanTuringInstitute,TheCentreforLong-TermResilience,CentrefortheGovernanceofAI,andUKAISafetyInstitute.AlsoaspecialthankstoDanHendrycks,DylanHadfield-Menell,andPamelaSamuelson.3InternationalScientificReportontheSafetyofAdvancedAI:InterimReport?Crowncopyright2024ThispublicationislicensedunderthetermsoftheOpenGovernmentLicencev3.0exceptwhereotherwisestated.Toviewthislicence,visit.uk/doc/open-government-licence/version/3orwritetotheInformationPolicyTeam,TheNationalArchives,Kew,LondonTW94DU,oremail:psi@.ukWherewehaveidentifiedanythird-partycopyrightinformationyouwillneedtoobtainpermissionfromthecopyrightholdersconcerned.Anyenquiriesregardingthispublicationshouldbesenttousat:secretariat.AIStateofScience@.ukDisclaimerThereportdoesnotrepresenttheviewsoftheChair,anyparticularindividualinthewritingoradvisorygroups,noranyofthegovernmentsthathavesupporteditsdevelopment.ThisreportisasynthesisoftheexistingresearchonthecapabilitiesandrisksofadvancedAI.TheChairofthereporthasultimateresponsibilityforit,andhasoverseenitsdevelopmentfrombeginningtoend.Researchseriesnumber:DSIT2024/0094InternationalScientificReportontheSafetyofAdvancedAI:InterimReportForewords 7ExecutiveSummary 91 Introduction 152 Capabilities 182.1 HowdoesGeneral-PurposeAIgainitscapabilities? 182.2 Whatcurrentgeneral-purposeAIsystemsarecapableof 192.2.1 Capabilitiesbymodality 202.2.2 Capabilitiesandlimitationsbyskill 212.3 Recenttrendsincapabilitiesandtheirdrivers 222.3.1 Recenttrendsincompute,data,andalgorithms 222.3.2 Recenttrendsincapabilities 252.4 Capabilityprogressincomingyears 292.4.1 Ifresourcescontinuetobescaledrapidly,wouldthisleadtorapidadvancements? 302.4.2 Willresourcesbescaledrapidly? 302.4.3 Willalgorithmicprogressleadtorapidadvancements? 323 Methodologytoassessandunderstandgeneral-purposeAIsystems 343.1 General-purposeAIassessmentsservetoevaluatemodelcapabilitiesandimpacts. 343.2 Approachesformodelperformanceanalysis 353.2.1 Casestudies 353.2.2 Benchmarks 353.2.3 Red-teamingandadversarialattacks 363.2.4 Auditing 373.3 Modeltransparency,explanations,andinterpretations 383.4 Challengeswithstudyinggeneral-purposeAIsystems 394 Risks 414.1 Malicioususerisks 414.1.1 Harmtoindividualsthroughfakecontent 414.1.2 Disinformationandmanipulationofpublicopinion 424.1.3 Cyberoffence 444.1.4 Dualusesciencerisks 454.2 Risksfrommalfunctions 474.2.1 Risksfromproductfunctionalityissues 474.2.2 Risksfrombiasandunderrepresentation 494.2.3 Lossofcontrol 514.3 Systemicrisks 544.3.1 Labourmarketrisks 544.3.2 GlobalAIdivide 574.3.3 Marketconcentrationrisksandsinglepointsoffailure 584.3.4 Riskstotheenvironment 594.3.5 Riskstoprivacy 605InternationalScientificReportontheSafetyofAdvancedAI:InterimReport4.3.6 Copyrightinfringement 614.4 Cross-cuttingriskfactors 634.4.1 Cross-cuttingtechnicalriskfactors 634.4.2 Cross-cuttingsocietalriskfactors 665 Technicalapproachestomitigaterisks 685.1 Riskmanagementandsafetyengineering 685.1.1 Riskassessment 695.1.2 Riskmanagement 705.2 Trainingmoretrustworthymodels 725.2.1 Aligninggeneral-purposeAIsystemswithdeveloperintentions 725.2.2 Reducingthehallucinationoffalsehoods 745.2.3 Improvingrobustnesstofailures 745.2.4 Removinghazardouscapabilities 755.2.5 Analysingandeditingtheinnerworkingsofmodels 755.3 Monitoringandintervention 765.3.1 Detectinggeneral-purposeAI-generatedcontent 765.3.2 Detectinganomaliesandattacks 775.3.3 Explainingmodelactions 775.3.4 BuildingsafeguardsintoAIsystems 775.4 Technicalapproachestofairnessandrepresentationingeneral-purposeAIsystems 785.4.1 Mitigationofbiasanddiscriminationworksthroughoutthestagesofgeneral-purposeAIdevelopmentanddeployment 795.4.2 Isfairnessingeneral-purposeAIsystemsachievable? 805.4.3 Challengesinachievingfairgeneral-purposeAIsystems 815.5 Privacymethodsforgeneral-purposeAIsystems 816 Conclusion 83Chair’snoteontheinterimreport 84Differingviews 86Glossary 87References 916InternationalScientificReportontheSafetyofAdvancedAI:InterimReportForewordsThisreportisthebeginningofajourneyonAISafetyIamhonouredtobechairingthedeliveryoftheinauguralInternationalScientificReportonAdvancedAISafety.IamproudtopublishthisinterimreportwhichistheculminationofhugeeffortsbymanyexpertsoverthesixmonthssincetheworkwascommissionedattheBletchleyParkAISafetySummitinNovember2023.WeknowthatadvancedAIisdevelopingveryrapidly,andthatthereisconsiderableuncertaintyoverhowtheseadvancedAIsystemsmightaffecthowweliveandworkinthefuture.AIhastremendouspotentialtochangeourlivesforthebetter,butitalsoposesrisksofharm.Thatiswhyhavingthisthoroughanalysisoftheavailablescientificliteratureandexpertopinionisessential.Themoreweknow,thebetterequippedwearetoshapeourcollectivedestiny.Ourmissionisclear:todriveashared,science-based,up-to-dateunderstandingofthesafetyofadvancedAI,andtocontinuetodevelopthatunderstandingovertime.ThereportrightlyhighlightsthatthereareareasofconsensusamongexpertsandalsodisagreementsoverthecapabilitiesandrisksofadvancedAI,especiallythoseexpectedtobedevelopedinthefuture.Inordertomeetourmissioneffectively,wehaveaimedtoaddressdisagreementamongsttheexpertcommunitywithintellectualhonesty.Bydissectingthesedifferences,wepavethewayforinformedpolicy-makingandstimulatetheresearchneededtohelpclearthefogandmitigaterisks.IamgratefultoourinternationalExpertAdvisoryPanelfortheirinvaluablecomments,initiallyshapingthereport’sscopeandlaterprovidingfeedbackonthefulldraft.Theirdiverseperspectivesandcarefulreviewhavebroadenedandstrengthenedthisinterimreport.Equallydeservingofrecognitionaremydedicatedteamofwritersandsenioradvisers.Theircommitmentoverthepastfewmonthshascreatedaninterimproductthathassurpassedmyexpectations.MythanksalsogototheUKGovernmentforstartingthisprocessandofferingoutstandingoperationalsupport.ItwasalsoimportantformethattheUKGovernmentagreedthatthescientistswritingthisreportshouldhavecompleteindependence.Thisinterimreportisonlythebeginningofajourney.Therearenodoubtperspectivesandevidencethatthisreporthasfailedtocaptureinthisfirstattempt.Inascientificprocesssuchasthis,feedbackisprecious.Wewillincorporateadditionalevidenceandscientificviewpointsasweworktowardthefinalversion.ProfessorYoshuaBengioUniversitédeMontréal/Mila-QuebecAIInstitute&Chair7InternationalScientificReportontheSafetyofAdvancedAI:InterimReportAISafetyisasharedglobalissueIamdelightedtopresentthisinterimupdateonthefirstInternationalScientificReportontheSafetyofAdvancedAI,akeyoutcomeofthegroundbreakingAISafetySummitheldatBletchleyParkinNovember2023.Thislandmarkreportrepresentsanunprecedentedglobalefforttobuildashared,science-basedunderstandingoftheopportunitiesandrisksposedbyrapidadvancementsinAI,andisatestamenttothe"BletchleyEffect"-thepowerofconveningbrilliantmindstotackleoneofhumanity'sgreatestchallenges.WebelievethatrealisingtheimmensepotentialofAItobenefithumanitywillrequireproactiveeffortstoensurethesepowerfultechnologiesaredevelopedanddeployedsafelyandresponsibly.Noonecountrycantacklethischallengealone.ThatiswhyIwassopassionateaboutbringingtogetheradiversegroupofworld-leadingexpertstocontributetheirknowledgeandperspectives.IwanttoespeciallythankProfessorYoshuaBengioforhisleadershipasChairinskilfullyshepherdingthiscomplexinternationaleffort.Cruciallythereportalsoshinesalightonthesignificantgapsinourcurrentknowledgeandthekeyuncertaintiesanddebatesthaturgentlyrequirefurtherresearchanddiscussion.Itismysincerehopethatthisreport,andthecooperativeprocessbehindit,canserveasacatalystfortheresearchandpolicyeffortsneededtoclosecriticalknowledgegapsandavaluableinputforthechallengingpolicychoicesthatlieahead.Westillhavemuchtolearn,butthisreportmarksanimportantstart.TheUKlooksforwardtocontinuingtoworkwithinternationalpartnerstopromotearesponsible,human-centricapproachtoAIdevelopment-onethatharnessesthesepowerfultoolstoimprovelivesandlivelihoodswhilevigilantlysafeguardingagainstdownsiderisksandharms.Together,wecanworktobuildafutureinwhichallofhumanitycanbenefitfromthewondersofAI.TheRtHonMichelleDonelanMP,SecretaryofState,DepartmentforScience,Innovation,andTechnologyAcriticalstepforwardandaCalltoActiononAISafetyTherapidadvancementofAIstandspoisedtoreshapeourworldinwaysbothprofoundandunforeseen.Fromrevolutionisinghealthcareandtransportationtoautomatingcomplextasksandunlockingscientificbreakthroughs,AI'spotentialforpositiveimpactisundeniable.However,alongsidethesenotablepossibilitiesliesignificantchallengesthatnecessitateaforward-lookingapproach.Concernsrangefromunintendedbiasesembeddedinalgorithmstothepossibilityofautonomoussystemsexceedinghumancontrol.Thesepotentialriskshighlighttheurgentneedforaglobalconversationtoensurethesafe,andresponsibleadvancementofAI.Inthiscontext,theInternationalAISafetyReportwillprovidevitalgroundworkforglobalcollaboration.Thereportrepresentsaconvergenceofknowledgefromexpertsacross30countries,theEuropeanUnion,andtheUnitedNations,providingacomprehensiveanalysisofAIsafety.ByfocusingontheearlyscientificunderstandingofcapabilitiesandrisksfromgeneralpurposeAIandevaluatingtechnicalmethodsforassessingandmitigatingthem,thereportwillsparkongoingdialogueandcollaborationamongmulti-stakeholders.Ihopethatbasedonthisreport,expertsfrom30countries,theEU,andtheUNcontinuetoengageinbalanceddiscussions,achievingAIriskmitigationthatisacceptableandtailoredtothespecificcontextofbothdevelopedanddevelopingcountries,therebycreatingafuturewhereinnovationandresponsibleAIcoexistharmoniously.LeeJong-Ho,MinisterofMSIT,RepublicofKorea8InternationalScientificReportontheSafetyofAdvancedAI:InterimReportExecutiveSummaryAboutthisreportThisistheinterimpublicationofthefirst‘InternationalScientificReportontheSafetyofAdvancedAI’.Adiversegroupof75artificialintelligence(AI)expertscontributedtothisreport,includinganinternationalExpertAdvisoryPanelnominatedby30countries,theEuropeanUnion(EU),andtheUnitedNations(UN).LedbytheChairofthisreport,theindependentexpertswritingthisreportcollectivelyhadfulldiscretionoveritscontent.AtatimeofunprecedentedprogressinAIdevelopment,thisfirstpublicationrestrictsitsfocustoatypeofAIthathasadvancedparticularlyrapidlyinrecentyears:General-purposeAI,orAIthatcanperformawidevarietyoftasks.Amidrapidadvancements,researchongeneral-purposeAIiscurrentlyinatimeofscientificdiscoveryandisnotyetsettledscience.Peoplearoundtheworldwillonlybeabletoenjoygeneral-purposeAI’smanypotentialbenefitssafelyifitsrisksareappropriatelymanaged.Thisreportfocusesonidentifyingtheserisksandevaluatingtechnicalmethodsforassessingandmitigatingthem.Itdoesnotaimtocomprehensivelyassessallpossiblesocietalimpactsofgeneral-purposeAI,includingitsmanypotentialbenefits.Forthefirsttimeinhistory,thisinterimreportbroughttogetherexpertsnominatedby30countries,theEU,andtheUN,andotherworld-leadingexperts,toprovideasharedscientific,evidence-basedfoundationfordiscussionsanddecisionsaboutgeneral-purposeAIsafety.Wecontinuetodisagreeonseveralquestions,minorandmajor,aroundgeneral-purposeAIcapabilities,risks,andriskmitigations.Butweconsiderthisprojectessentialforimprovingourcollectiveunderstandingofthistechnologyanditspotentialrisks,andformovingclosertowardsconsensusandeffectiveriskmitigationtoensurepeoplecanexperiencethepotentialbenefitsofgeneral-purposeAIsafely.Thestakesarehigh.Welookforwardtocontinuingthiseffort.HighlightsoftheexecutivesummaryIfproperlygoverned,general-purposeAIcanbeappliedtoadvancethepublicinterest,potentiallyleadingtoenhancedwellbeing,moreprosperity,andnewscientificdiscoveries.However,malfunctioningormaliciouslyusedgeneral-purposeAIcanalsocauseharm,forinstancethroughbiaseddecisionsinhigh-stakessettingsorthroughscams,fakemedia,orprivacyviolations.Asgeneral-purposeAIcapabilitiescontinuetoadvance,riskssuchaslarge-scalelabourmarketimpacts,AI-enabledhackingorbiologicalattacks,andsocietylosingcontrolovergeneral-purposeAIcouldemerge,althoughthelikelihoodofthesescenariosisdebatedamongresearchers.Differentviewsontheserisksoftenstemfromdifferingexpectationsaboutthestepssocietywilltaketolimitthem,theeffectivenessofthosesteps,andhowrapidlygeneral-purposeAIcapabilitieswillbeadvanced.Thereisconsiderableuncertaintyabouttherateoffutureprogressingeneral-purposeAIcapabilities.Someexpertsthinkaslowdownofprogressisbyfarmostlikely,whileotherexpertsthinkthatextremelyrapidprogressispossibleorlikely.Therearevarioustechnicalmethodstoassessandreducerisksfromgeneral-purposeAIthatdeveloperscanemployandregulatorscanrequire,buttheyallhavelimitations.Forexample,currenttechniquesforexplainingwhygeneral-purposeAImodelsproduceanygivenoutputareseverelylimited.9InternationalScientificReportontheSafetyofAdvancedAI:InterimReportThefutureofgeneral-purposeAItechnologyisuncertain,withawiderangeoftrajectoriesappearingpossibleeveninthenearfuture,includingbothverypositiveandverynegativeoutcomes.ButnothingaboutthefutureofAIisinevitable.ItwillbethedecisionsofsocietiesandgovernmentsthatwilldeterminethefutureofAI.Thisinterimreportaimstofacilitateconstructivediscussionaboutthesedecisions.Thisreportsynthesisesthestateofscientificunderstandingofgeneral-purposeAI–AIthatcanperformawidevarietyoftasks–withafocusonunderstandingandmanagingitsrisksThecapabilitiesofsystemsusingAIhavebeenadvancingrapidly.ThishashighlightedthemanyopportunitiesthatAIcreatesforbusiness,research,government,andprivatelife.IthasalsoledtoanincreasedawarenessofcurrentharmsandpotentialfuturerisksassociatedwithadvancedAI.ThepurposeoftheInternationalScientificReportontheSafetyofAdvancedAIistotakeasteptowardsasharedinternationalunderstandingofAIrisksandhowtheycanbemitigated.ThisfirstinterimpublicationofthereportrestrictsitsfocustoatypeofAIwhosecapabilitieshaveadvancedparticularlyrapidly:general-purposeAI,orAIthatcanperformawidevarietyoftasks.Amidrapidadvancements,researchongeneral-purposeAIiscurrentlyinatimeofscientificdiscoveryandisnotyetsettledscience.Thereportprovidesasnapshotofthecurrentscientificunderstandingofgeneral-purposeAIanditsrisks.Thisincludesidentifyingareasofscientificconsensusandareaswheretherearedifferentviewsoropenresearchquestions.Peoplearoundtheworldwillonlybeabletoenjoythepotentialbenefitsofgeneral-purposeAIsafelyifitsrisksareappropriatelymanaged.Thisreportfocusesonidentifyingrisksfromgeneral-purposeAIandevaluatingtechnicalmethodsforassessingandmitigatingthem,includingthebeneficialuseofgeneral-purposeAItomitigaterisks.Itdoesnotaimtocomprehensivelyassessallpossiblesocietalimpactsofgeneral-purposeAI,includingwhatbenefitsitmayoffer.General-purposeAIcapabilitieshavegrownrapidlyinrecentyearsaccordingtomanymetrics,andthereisnoconsensusonhowtopredictfutureprogress,makingawiderangeofscenariosappearpossibleAccordingtomanymetrics,general-purposeAIcapabilitiesareprogressingrapidly.Fiveyearsago,theleadinggeneral-purposeAIlanguagemodelscouldrarelyproduceacoherentparagraphoftext.Today,somegeneral-purposeAImodelscanengageinmulti-turnconversationsonawiderangeoftopics,writeshortcomputerprograms,orgeneratevideosfromadescription.However,thecapabilitiesofgeneral-purposeAIaredifficulttoestimatereliablyanddefineprecisely.Thepaceofgeneral-purposeAIadvancementdependsonboththerateoftechnologicaladvancementsandtheregulatoryenvironment.Thisreportfocusesonthetechnologicalaspectsanddoesnotprovideadiscussionofhowregulatoryeffortsmightaffectthespeedofdevelopmentanddeploymentofgeneral-purposeAI.AIdevelopershaverapidlyadvancedgeneral-purposeAIcapabilitiesinrecentyearsmostlybycontinuouslyincreasingresourcesusedfortrainingnewmodels(atrendcalled‘scaling’)andrefiningexistingalgorithms.Forexample,state-of-the-artAImodelshaveseenannualincreasesofapproximately4xincomputationalresources(‘compute’)usedfortraining,2.5xintrainingdatasetsize,and1.5-3xinalgorithmicefficiency(performancerelativetocompute).Whether‘scaling’hasresultedinprogressonfundamentalchallengessuchascausalreasoningisdebatedamongresearchers.10本報(bào)告來(lái)源于三個(gè)皮匠報(bào)告站（）,由用戶(hù)Id:673421下載,文檔Id:464666,下載日期:2025-01-24InternationalScientificReportontheSafetyofAdvancedAI:InterimReportThepaceoffutureprogressingeneral-purposeAIcapabilitieshassubstantialimplicationsformanagingemergingrisks,butexpertsdisagreeonwhattoexpecteveninthenearfuture.Expertsvariouslysupportthepossibilityofgeneral-purposeAIcapabilitiesadvancingslowly,rapidly,orextremelyrapidly.Thisdisagreementinvolvesakeyquestion:willcontinued‘scaling’ofresourcesandrefiningexistingtechniquesbesufficienttoyieldrapidprogressandsolveissuessuchasreliabilityandfactualaccuracy,orarenewresearchbreakthroughsrequiredtosubstantiallyadvancegeneral-purposeAIabilities?Severalleadingcompaniesthatdevelopgeneral-purposeAIarebettingon‘scaling’tocontinueleadingtoperformanceimprovements.Ifrecenttrendscontinue,bytheendof2026somegeneral-purposeAImodelswillbetrainedusing40xto100xmorecomputethanthemostcompute-intensivemodelspublishedin2023,combinedwithtrainingmethodsthatusethiscompute3xto20xmoreefficiently.However,therearepotentialbottleneckstofurtherincreasingbothdataandcompute,includingtheavailabilityofdata,AIchips,capitalexpenditure,andlocalenergycapacity.Companiesdevelopinggeneral-purposeAIareworkingtonavigatethesepotentialbottlenecks.Severalresearcheffortsaimtounderstandandevaluategeneral-purposeAImorereliably,butouroverallunderstandingofhowgeneral-purposeAImodelsandsystemsworkislimitedApproachestomanagingrisksfromgeneral-purposeAIoftenrestontheassumptionthatAIdevelopersandpolicymakerscanassessthecapabilitiesandpotentialimpactsofgeneral-purposeAImodelsandsystems.Butwhiletechnicalmethodscanhelpwithassessment,allexistingmethodshavelimitationsandcannotprovidestrongassurancesagainstmostharmsrelatedtogeneral-purposeAI.Overall,thescientificunderstandingoftheinnerworkings,capabilities,andsocietalimpactsofgeneral-purposeAIisverylimited,andthereisbroadexpertagreementthatitshouldbeaprioritytoimproveourunderstandingofgeneral-purposeAI.Someofthekeychallengesinclude:Developersstillunderstandlittleabouthowtheirgeneral-purposeAImodelsoperate.Thisisbecausegeneral-purposeAImodelsarenotprogrammedinthetraditionalsense.Instead,theyaretrained:AIdeveloperssetupatrainingprocessthatinvolvesalotofdata,andtheoutcomeofthattrainingprocessisthegeneral-purposeAImodel.Thesemodelscanconsistoftrillionsofcomponents,calledparameters,andmostoftheirinnerworkingsareinscrutable,includingtothemodeldevelopers.Modelexplanationandinterpretabilitytechniquescanimproveresearchers’anddevelopers’understandingofhowgeneral-purposeAImodelsoperate,butthisresearchisnascent.General-purposeAIismainlyassessedthroughtestingthemodelorsystemonvariousinputs.Thesespotchecksarehelpfulforassessingstrengthsandweaknesses,includingvulnerabilitiesandpotentiallyharmfulcapabilities,butdonotprovidequantitativesafetyguarantees.Thetestsoftenmisshazardsandoverestimateorunderestimatecapabilitiesbecausegeneral-purposeAIsystemsmaybehavedifferentlyindifferentcircumstances,withdifferentusers,orwithadditionaladjustmentstotheircomponents.Independentac