[Th %SSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK J r J r SSK J r SSKJrJr SSKJr SSKJrJrJrJrJrJrJrJr SSKrSSKJr SSKJ r SS K!J"r"J#r# SS K$J%r% \RL"\'5r(S \)4S jr*S \)4S jr+S \)4Sjr,"SS\5r-\"S5r."SS5r/S \)4Sjr0\0"5(aSS/r1OS/r1"SS\ 5r2\1SSS.SS0SS0SS000S.0000SS0S./S S!/S S!/\%"5/S".r3\4\5\6\4\7S#'\\5\8\\/\4r9"S$S%\ 5r:"S&S'5r;\;"5r<\=\5S(4r>"S)S*5r?\4\5\4r@\/\)4rA\/\A4rB0S+S_S,\<_S-\<_S.\<_S/\<_S0\<_S1\<_S2\<_S3\<_S4\<_S5\<_S6\<_S7\<_S8\<_S9\<_S:\<_S;\<_0S<\<_S=\<_S>\<_S?\<_S@\<_SA\<_SB\<_SC\<_SD\<_SE\<_SF\<_SG\<_SH\<_SI\<_SJ\<_SK\<_SL\<_E\<\<\<\<\<\<\ This class returns examples of a Type, given its class name. NTYPE_EXEMPLARStrcV[RRURS5$)z Return an example of a class. N)r>r@getr#rAs rexampleTypeExemplars.example[s! ++// DAArc<UR[R;$r*)r#r>r@rDs rcontainsTypeExemplars.containsbszz]9999rr+)r#r4r5r6r7rr&r8r9r(r:rr@dictstrr __annotations__ staticmethodrr<rrEboolrHr;r+rrr>r>Ps   )+ %%uxx~~';';'=""$5d$;&NDcN B47Bx{BB :DG:::rr>cR[R"S5 g![a gf=f)z"checks if we have halide availablehalideTF) importlib import_moduleModuleNotFoundErrorr+rrcheck_halide_importrTgs*) s  &&tritonrPc>\rSrSrSrSrSrSrSrSr Sr S \ 4S jr S r g ) Statusvz0 The Status return value enum for Config Fuzzer skippedpassedfailed_compilefailed_run_compile_exceptionfailed_run_eager_exceptionfailed_run_returnrcU[R:H=(dG U[R:H=(d- U[R:H=(d U[R:H$)zE Convenience method to check whether these status represent failure. )rWFAILED_COMPILEFAILED_RUN_EAGER_EXCEPTIONFAILED_RUN_COMPILE_EXCEPTIONFAILED_RUN_RETURNr1s rfailingStatus.failingsR F)) ) 0v888 0v::: 0v///  rr+N)r#r4r5r6r7SKIPPEDPASSEDr`rbrarcrNrdr;r+rrrWrWvs8 G F%N#A !=,    rrWT)shape_broadcast_batch_linearfuse_nodes_with_same_usersfuse_nodes_with_same_parentF)batch_linear_post_gradbatch_aten_mulbatch_aten_sigmoidbatch_aten_addnormalization_aten_passunbind_stack_aten_passrequire_fbgemm)rnrlbatch_aten_subbatch_aten_div group_linearpad_mmmixed_mm) cuda_backendpost_grad_fusion_optionsautoheuristic_collectautoheuristic_usetraceable_tensor_subclassesTYPE_OVERRIDESc j\rSrSrSrSrSr\S\S\ S\ \ S\ S \ 4 S j5r \S SS \ 4S j5rS rg)SamplingMethoda  This class handles the process of assigning concrete values to type annotations. So a type annotation of ```python foo: Optional[int] = None ``` Will be assigned an int if the dispatch function gets TOGGLE, or a 50/50 split between an int and None if it gets RANDOM. TOGGLERANDOM random_sample field_namerdefaultrc H^^^^T[;a[R"[T5$U[:Xa&T(a[R"SS/5$U(+$U[:Xa[R "SS5$U[ :Xa[R"SS5$U[:Xak[R[R-[R-mSRU4Sj[[R "SS5555$[U[ 5(a[#US [%U5(a['US5/O ['S 5/5Sn[%U5S:aUSOS n[[R "SS 55Vs/sHn[(R+TTXE5PM sn$[U[,5(a[!U5n[#US [%U5(a['US5/O ['S 5/5Sn[%U5S:aUSOS n[[R "SS 55Vs1sHn[(R+TTXE5iM sn$[U[.5(a[!U5n[#US [%U5(a['US5/O ['S 5/5Sn[%U5S:aUSOS n[/[[R "SS 55Vs/sHn[(R+TTXE5PM sn5$[U[05(Ga [#US Ub@[%U5(a0[3[&[5[7UR95555O['S 5['S 545upUbB[%UR955S:a%[5[7UR9555upOS up[[R "SS 55Vs0sH0n[(R+TTX5[(R+TTX5_M2 sn$[U[:5(a[%UR<5S:deT(a![R"UR<5n OA[R"UR<Vs/sHo['U5:wdMUPM sn5n U "5n[(R+TTX5$[U[D5(aI[#US [E[3[&U555n[GX5n[E[3UU4SjU55$[U[H5(ajT(a [R"UR<5$UR<Vs/sH oU:wdM UPM nnU(a[R"U5$U$[KU5(a`UR<SnT(a-[R"S [(R+TTXC5/5$Uc[(R+TTUS 5$g U['S 5Lag [MU5(a2[!UR<5Sm[OS5UUU4Sj5nU$[PRSU5(a[PRUU5$U[V:Xa US:XdS$S$[ASUS35es snfs snfs snfs snf![>an [AS 5U eS n A ff=fs snf![Ba S nGNMf=fs snf![>an [AS5U eS n A ff=f![>an [AS5U eS n A ff=f![>an [AS5U eS n A ff=f)z3 Generates a value of a type based on the setting. TFrc3P># UHn[R"T5v M g7fr*)randomchoice).0_ characterss r :SamplingMethod._generate_value_for_type..s!3Oa j))3Os#&__args__N)NNzUnion type with no argsc@>[RTTUSUS5$)Nrrr~_generate_value_for_type)xrrs r9SamplingMethod._generate_value_for_type..7s!nEE%z1Q41rzLiteral type with no argszOptional type with no argszCallable type with no argscgr*r+)r kwargss rrrfs4rc4>[RTTTS5$r*r)r rrr return_types rdummy_function?SamplingMethod._generate_value_for_type..dummy_functionfs%>>!:{DrzUnable to process type z. PRs welcome :)),r|rrrNintrandintfloatuniformrKstring ascii_lettersdigits punctuationjoinrangerlistgetattrlenrr~rsetrrJmapnextiteritemsrrAttributeError ValueError Exceptiontuplezipr r!r$rr>rHrEr )rrrr elem_type new_defaultr indexablekey_type value_type default_key default_valerrnew_typerAr zippedchoicesrrrs`` @@rr'SamplingMethod._generate_value_for_types  '== !;< <  3@6==$/ Q'k Q # >>!T* * % >>!T* * # -- =@R@RRJ773829N3O Y % %&)'llgaj!"d  I ),G q(8'!*dK v~~a34 5A77!:y5   Y $ $W I(+G il#$4:, I +.g,*:)A,K v~~a34 5A77!:y5   Y + +W I(+G il#$4:, I +.g,*:)A,K #6>>!Q#78 9#;;%z99  Y % %#*'CLLD$tGMMO4564j$t*- $ H"s7==?';a'?+/W]]_0E+F( [+5( v~~a34  5A 77!:x!::!:z 5  Y & & E9--.222!==););<!== ) 2 2I 214=6HQ 2I #&j "::z8 Y & &c$()D 'F  Y ( ( G !==););<<*3*<*<M*CG H." H !=>CG Hs![;!\!\7\ '\ \-\--\2?&] &] 4 ]]] &] 9](8^ \* \%%\*2 ]]] ]% ]  ]%( ^2 ]>>^ ^! ^^!smcU[R:Xa[[RS5$U[R:Xa[[RS5$[ SU35e)zb Returns a function that will generate values from a type, based on the SamplingMethod passed in. TFzmalformed sampling method: )r~rrrrr)rs rdispatchSamplingMethod.dispatchtsY && &>BBDI I >(( (>BBEJ J:2$?@ @rr+N)r#r4r5r6r7rrrMrNrKrr r SamplingTyperr;r+rrr~r~sF FoToT),oT9=coTMPoT oToTb A% A, A Arr~c\rSrSrSrSrg)Defaultizn Singleton default object that will cause the ConfigFuzzer to always use the default value set in the config. r+N)r#r4r5r6r7r;r+rrrrsrr.c\rSrSr%Sr\\\4\S'S\ 4Sjr SSjr S\ 4Sjr S\ 4S jrS \S \SS4S jrS \S\\4S jrS\\4SjrSrg) ResultTypeizX The mapping of the combo strings to the result status after running the config fuzzer. _valsrc"SURS3$)Nz ResultType[]rr1s r__repr__ResultType.__repr__sTZZL**rNc0Ulgr*rr1s r__init__ResultType.__init__s  rc,[UR5$r*)rrr1s r__len__ResultType.__len__s4::rc[UR5nURR5HnU[R:XdMUS-nM U$)z9 Returns how many combos actually ran (weren't skipped). r)rrvaluesrWrf)r,retstatuss rnum_ranResultType.num_ransD$**ojj'')F'q* rcomborcH[[U55nX RU'gr*)rsortedr)r,rrs rrResultType.setsfUm$" 5rcb[[U55nURRUS5$r*)rrrrC)r,rs rlookupResultType.lookups%fUm$zz~~eT**rc6URR5$r*)rkeysr1s rrResultType.keysszz  rrrN)r#r4r5r6r7rJ ComboTyperWrLrKrrrrrrrrrrr;r+rrrrs  6! ""+#+##F#t#+I+(6*:+!hy)!rrforce_disable_cacheszcpp.cxx TYPE_CHECKINGmax_autotune_pointwisemax_autotune_gemmmax_autotune_gemm_backendsmax_autotune_conv_backendsmax_autotune_gemm_search_space+max_autotune_subproc_result_timeout_seconds-max_autotune_subproc_graceful_timeout_seconds.max_autotune_subproc_terminate_timeout_secondszaot_inductor.presetsz cuda.archz cuda.versionzcuda.cutlass_dirz cuda.cuda_cxxz rocm.archzrocm.ck_supported_archz rocm.ck_dirzrocm.rocm_home"check_stack_no_cycles_TESTING_ONLYsleep_sec_TESTING_ONLYz#triton.inject_relu_bug_TESTING_ONLY reorder_for_compute_comm_overlapenabled_metric_tablesztriton.debug_sync_graphztriton.debug_sync_kernelprofile_bandwidth_regexdisable_cpp_codegenztrace.save_real_tensorspre_grad_fusion_optionsexternal_matmulz'test_configs.autotune_choice_name_regexz'test_configs.autotune_choice_desc_regex)z'cpp.enable_floating_point_contract_flagpost_grad_custom_pre_passpost_grad_custom_post_pass'reorder_for_compute_comm_overlap_passesjoint_custom_post_passjoint_custom_pre_passpre_grad_custom_pass)r{!compiled_autograd_kwargs_overridefail_on_recompile_limit_hitsuppress_errors)ztorch._inductor.configztorch._dynamo.configMODULE_DEFAULTSc \rSrSr%Sr\\S'\\S'S\RS4S\ S\ S \ S\ \S \S \ 4 S jjrS \4SjrS\S\S S4SjrS.SjrS \4SjrS\\S \4SjrS\S\S S4SjrS\S \4SjrS\S\S \4SjrS/S\ S\ S \4SjjrS0S \S S4S!jjrS0S \S S4S"jjrS#\ S$\ \ S S4S%jr!S\S\S \4S&jr"S1S'\ S(\#S \$\4S)jjr%S\S*\S \ \4S+jr&S\S*\$\'\\4S \ \4S,jr(S-r)g)2 ConfigFuzzeria This tool makes it easy to search through config state-space with a minimal reproduction or test, either for debugging or just bug hunting. It has two entry points: - bisect, which randomly flips configs and tries to find the minimal reproduction upon failure. - fuzz_n_tuple, which tries every combination of n configs. This grows quickly as a function of n, so beware. bisect is recommended, but fuzz_n_tuple can give you peace of mind that a new config will compose with every other config. The main interface is a function factory that will return Callables to be torch.compiled. This function factory should return a test function when it's called. Said test function returns a boolean, which determines whether the ConfigFuzzer considers it a successful run or not. Throwing an exception from within the function will be considered a failure as well. # Example usage: ```python import torch._inductor.config as cfg def create_simple_test_model_gpu() -> FactoryOutputType: batch_size = 32 seq_length = 50 hidden_size = 768 def test_fn() -> bool: inp = torch.randn(batch_size, seq_length, hidden_size, device="cuda") weight = torch.randn(hidden_size, hidden_size, device="cuda") matmul_output = inp @ weight final_output = torch.nn.LayerNorm(hidden_size, device="cuda")(matmul_output) return True return test_fn fuzzer = ConfigFuzzer(cfg, create_simple_test_model_gpu, seed=2) # Test every pair of configs: results = fuzzer.fuzz_n_tuple(n, max_combinations=10000000) visualize_results(n, results) # Test random configs with bisection: ret = fuzzer.bisect(num_attempts=10) # reproduce a failing config fuzzer.reproduce( [{"triton.autotune_pointwise": ..., "coordinate_descent_tuning": ...}] ) ``` The list of known failures on inductor config are: cpp_wrapper, triton_debug_sync_graph cpp_wrapper, triton_debug_sync_kernel cpp_wrapper, disable_cpp_codegen combo_kernels, benchmark_combo_kernel, profile_bandwidth, profile_bandwidth_regex trace.enabled, trace.save_real_tensors samplerNi config_moduletest_model_fn_factoryseedr test_timeoutc[RS:a[RS5 gX0lX`l0UlXlX lURRUl [RU5Ul UcLURR[;a#[URRUlg[#S5eX@lg)a Args: config_module: The module containing the configs to fuzz test_model_fn_factory: Function that returns a test model, which runs and returns True if successful, or the outputs if they should be compared with eager seed: Randomness seed. default: Default values for the config. Inductor has preset based on know failures. sm: How type value samples are generated, default TOGGLE. test_timeout: max time a test can take. )r z$Only python 3.10 and later supportedNz"No default passed to ConfigFuzzer.)sys version_infologerrorr rdetailed_resultsr r _configfieldsr~rr r#rrr)r,r r r rrrs rrConfigFuzzer.__init__<s&   g % II< =  (AC*%:"/3/A/A/I/I $--b1 ?!!**o=.t/A/A/J/JK  !EFF"Lrrc pSURSURSURSURS3 $)NzConfigFuzzer(config_module=z, test_model_fn_factor=z, seed=z , default=))r r r rr1s rrConfigFuzzer.__repr__bsH)$*<*<)=>$$($>$>#?wtyykQ[\`\h\h[iij l rrvaluec0[URX5 g)z!Set a config value in the module.N)setattrr )r,rrs r _set_configConfigFuzzer._set_confighs""J6rcURR5H upURXR5 M" g)z*Reset all configs to their default values.N)rrrr)r,r field_objs r_reset_configsConfigFuzzer._reset_configsls0%)[[%6%6%8 !J   Z):): ;&9rcURR5VVs0sH*upX[:waUOURUR_M, nnnU$s snnf)z%creates a new config from the default)rrDEFAULTr)r,namevalrs r new_configConfigFuzzer.new_configqsY"\\//1 1  #T[[->-F-F F1    s1AconfigscN[5nUHnURX25 M U$)z#entrypoint to reproduce any failurer_reproduce_single_helper)r,r+resultsconfs r reproduceConfigFuzzer.reproduceys&,D  ) )$ 8rr0r/c [SU35 UR5nURU5 URX#5 [SUSUR [ UR 55535 g)NzStarting repro of z Status of z: )printr)update test_configrrr)r,r0r/r)s rr.%ConfigFuzzer._reproduce_single_helpersd "4&)*__& $ - 4&GNN53E$F#GHIrconfigc<[5nURX5 U$r*r-)r,r8r/s rreproduce_singleConfigFuzzer.reproduce_singles, %%f6rrc[U5 URU5=n(aU$UR5nSnUH[nXd;aSnURS5(aSnURUnUR XgR UR5nXU'M] U(a0URU[R5 [R$URX5$)NFTr) r4rr) startswithrr rrrrWrfr6) r,r/rstr8skiprfieldrs r _fuzz_helperConfigFuzzer._fuzz_helpers e & &2 &I"J#$$S))KK +EKK ,<,> !00rnmax_combinationsc`[5n[SUSUR35 [R"UR5 [R "UR U5HCnURX45nU[R:wdM*US-nUS::dM7[S5 U$ U$)zp Test every combination of n configs. returns a dict of this shape: {(config-1, config-2... config-n): status} z Starting z-tuple testing with seed rrz"Reached maximum combinations limit) rr4r r itertools combinationsrrArWrf)r,rCrDr/rr>s r fuzz_n_tupleConfigFuzzer.fuzz_n_tuples , !5dii[AB DII++DKK;E""72BV^^# A% #q(>?<rfilenamec[US5n[R"URURS.U5 SSS5 g!,(df  g=f)z'Save the current fuzzer state to a filewb)r/rN)openpickledumpr/r)r,rJfs r save_stateConfigFuzzer.save_states; (D !Q KK LLd>S>STVW " ! !s /A Ac[US5n[R"U5nUSUlUR S05UlSSS5 g!,(df  g=f)zLoad fuzzer state from a filerbr/rN)rMrNloadr/rCr)r,rJrPstates r load_stateConfigFuzzer.load_statesG (D !QKKNE +DL$)II.@"$ED !" ! !s 8A Asignumframec[S5e)NzTest execution timed out) TimeoutError)r,rYrZs rtimeout_handlerConfigFuzzer.timeout_handlers566rc ^^^^ ^ ^^[R"[RTR5m[R"TR5 [ ST35 [ TR55m TRT 5=n(a'[R"[RT5 U$SU4SjjmS[S[[[44U4Sjjm S[S[S [S[[S[4 U U UUUU4S jjn[ R"R%5 TR'5 TR)5HupVTR+XV5 M TR-5nU"5 TR-5n [ R0"U S S9n U "5n U (dU"S[R6SS5$U"S[R8SS5$![a#nU"S [R.S U5sSnA$SnAff=f![a#nU"S[R2S U5sSnA$SnAff=f![a#nU"S[R4S U5sSnA$SnAff=f)zJ Tests a config by calling the function produced by the factory function. zTesting config rNcX>TR5Hup[USU35 M g)Nz = )rr4)r@rr8s r print_config.ConfigFuzzer.test_config..print_configs( &  s5'*+!/rexccd>[U5[R"5TR5S.$)N) exception tracebackr8)rKrf format_exccopy)rcr8s rget_error_info0ConfigFuzzer.test_config..get_error_infos) X&113 ++- rmessage return_statusprint_tracebackc>[R"[RT5 [US35 T"5 U(aT"U5T RT'U(a[R "5 TR TU5 U$)Nz with config combination:)signalSIGALRMr4rrf print_excr) rkrlrmrc config_tuplerioriginal_handlerrar/r,s r handle_return/ConfigFuzzer.test_config..handle_returnsh MM&..*: ; WI67 8 N6DS6I%%l3##% KK m 4 rzEager exceptionTinductor)backendzException compilingzException running compiledzFunction returned FalseFzFunction succeededr)rorpr]alarmrr4rrrrrJrKr rWrNrr8_dynamoresetr#rrr racompiler`rbrcrg)r,r/r8rrtr'r test_model_fnrctest_model_fn2compcompile_resultrrrirsras``` @@@@rr6ConfigFuzzer.test_configs%"==9M9MN T&&' x()V[[]+ ... .3 . MM&..*: ;J ,   d38n  ! !! !" !)$  !   ! !"   !<<>KD   T )*224   O !779N==DD !VN )6+C+CUD !!5v}}eTR R?  !6#D#DdC    %v'<'[U5nTRTU5nU$r*)rJr6)rdresultr/r,s rtest8ConfigFuzzer._bisect_failing_config_helper..testRs#QA%%gq1FMrrr)r4rrrKr rWrrdrJrshufflerr enumerate)r,r/rrmid first_half second_halflowhighquart1quart2quart3quart4ijrnew_lists`` rr*ConfigFuzzer._bisect_failing_config_helperEsO ">"234 DsCx) f   ~ ! #+/+?+G+G+I+I4' St S~&.!Q&#DS) $T*   # # % %55gzJ J   $ $ & &55g{K K ~ ! #n%*C9D#D)FF|##%%99'6JJ#DS)K7FF|##%%99'6JJ$"77FF|##%%99'6JJ#E*FF|##%%99'6JJs>*+A&/&?J&?da16&?HJH~%%''99'8LL, N## Ks  I' I') r rrrr/r r r rr)r)zfuzzer_state.pkl)dg?)*r#r4r5r6r7rrL ConfigTyper~rr FactoryTyperrrrKrr rr#r)rrr1r.r:rrWrArHrQrWrr]r6rrrrrrr;r+rrr r s9v  )-+22 $##$# +$# $# *% $#  $#$#L # 7c7#7$7< J*!5*JZJ*JQUJzj 1J1y1V10cSJ(3F3FF7c7(92E7$7PS:PSzPSfPSd#3##jAQ#JY!Y3=Y * Y 4$!4$37c3h3H4$ * 4$rr rCr/rJcnUS:Xde[U5S:de[05nUR5H+nURUS5 URUS5 M- [ U5nSnUS- nUHnSR U5nUSUS3- nM US - nUHn US U S3- nUHnUR X45n S n S n U [R:XaS n S n OU [R:XaSn Sn OhU [R:XaSn Sn OOU [R:XaSn Sn O6U [R:XaSn Sn OU [R:XaSn Sn OS n S n USU SU S3- nM US- nM US- n[US5n U RU5 SSS5 g!,(df  g=f)zh Creates an HTML document representing the results of running the fuzzer with fuzz_n_tuple, with n = 2. rrra Fuzzer Visualization

Fuzzer Visualization

zz
zzzzz?
\z
rrY-rZOfailedeERCz z
wN)rrraddrrrrWrfrgrarbrcr`rMwrite)rCr/rJ input_setkey input_list html_contentcol_namecolrow_name status_enum status_class status_valfiles rvisualize_resultsr|s 6M6 wKLJfnn,(   -'   A AA'   C CC'   8 88'   5 55'  (  k,r*UK KL7#8  =@L h  <    s F&& F4)z results.html)GrQrFloggingrNrrorrrfcollections.abcrrenumr functoolsrrtypesrtypingr r r r r rrrr8!torch._inductor.custom_graph_passrtorch._inductor.schedulerrtorch.utils._config_modulerrtorch.utils._ordered_setr getLoggerr#rrNrr!r$r&r<r>rT CUDA_BACKENDrWr|rJrKrrLrrr~rr&rrrrFactoryOutputTyperrr rrr+rrrs .$    =7A/!HTH 4 ,4,   CL::.Th'L:L T F!15.2' =eD#@$"G1(S$s)^$4d3i-s23 IATIAX ) #s(O #!#!N#s(^ RX&r,,- "**7* * !' * W * %g * %g* )'* 6w* 8* 9'* * W* * G* !*" W#*$ !'%*& w'*( ')** -g+*, !'-*. .w/*0 +G1*2 3*4 "75*6 #G7*8 "79*: w;*< "7=*> "7?*@ 7A*B 27C*D 27E*F4;%,&-3:")!( 'S*X(/-4'." Y2*c:o&2j|$|$@ 2@h! h!h!+.h! h!r