[Th"<SrSSKrSSKrSSKrSSKrSSKrSSKrSSKJr SSK J r J r J r J r Jr SSKJr SSKJr SSKrSSKJrJr SS KJrJrJr SS KJrJrJrJrJ r J!r!J"r"J#r#J$r$J%r%J&r& SS K'J(r(J)r) SS K*J+r+J,r,J-r-J.r. SS K/J0r0J1r1J2r2J3r3J4r4J5r5J6r6J7r7J8r8 SSK9J:r:J;r;Jr> SSK?J@r@ \ (aSSKBJCrC SSKDJErEJFrF \"S\ S9rGSQSRSjjrHSRSjrIS\J\K\<44SjrLSrM1SkrNSrO"SS\<5rP"SS \P5rQ"S!S"\P5rR"S#S$\<5rS"S%S&\S5rT"S'S(\P5rU"S)S*\U5rV"S+S,\Q5rW"S-S.\W5rX"S/S0\Q5rYSRS1jrZ"S2S3\P5r["S4S5\<5r\"S6S7\<5r]S8r^SRS9jr_"S:S;\Q5r`"S<S=\Q5ra"S>S?\Q5rb"S@SA\<5rc"SBSC\<5rd"SDSE\<5re"SFSG\<5rfSSHKDJgrgJhrh "SISJ\h5ri\i"5rj"SKSL\<5rk"SMSN\<5rl"SOSP\<5rmg!\Aa Sr@GNnf=f)Sa Function-related variable tracking classes for Dynamo's symbolic execution. This module contains classes that track different types of functions during graph compilation, including: - User-defined functions and methods - Built-in functions and methods - Wrapped functions (e.g. from decorators) - Special function types (e.g. functools.partial) - Triton kernels and related function types These classes are responsible for: - Tracking function calls and their arguments - Managing function closures and cell variables - Handling function attributes and special methods - Maintaining guards for function identity and closure contents - Supporting function inlining and specialization - Enabling proper symbolic execution of different function types The variable trackers here work together with the rest of Dynamo to enable accurate graph capture while handling Python's various function-related behaviors. N)Sequence)AnyCallableOptional TYPE_CHECKINGTypeVar)Never)patch) polyfills variables)create_call_function create_rot_n is_generator) get_dynamo_observed_exceptionhandle_observed_exceptionInfiniteGeneratorErrorObservedExceptionObservedGeneratorExitObservedUserStopIterationraise_observed_exception SkipFrame unimplementedunimplemented_v2 Unsupported) GuardBuilder install_guard) AttrSourceConstantSourceDefaultsSource GetItemSource) check_constant_argscheck_unspec_or_constant_argscmp_name_to_op_mappingcountersidentity is_functionis_wrapper_or_member_descriptoristype make_cell)typestrValueMutationNewVariableTrackerConstantVariable)_fsdp_param_group)InstructionTranslator)TritonGridTypeTritonKernelType_F)boundc[U[5(aU$U(d[R"X5$[RR X5$N) isinstancer.buildr LazyVariableTrackercreate)txvalsources Y/var/www/auris/envauris/lib/python3.13/site-packages/torch/_dynamo/variables/functions.pywrap_bound_argrAXsA#'' $$R--,,33C@@c[UR55H/up#[U[[45(dM"[ X5X'M1 gr8)listitemsr9tupledictrA)r=resultkvs r@wrap_args_kwargsrKds6V\\^$ a% ' '&r-FI%rBrHcURRnURH<nUR5nXA;a UR XQR U55 XQU'M> g)z Update `result` to add mapping from local name to new cells created directly by `code`, or update SideEffects in `parent` if the a local cell is already in `result` (cell argument). N)output side_effects co_cellvarstrack_cell_new store_cellpop)parentrHcoderNnamenew_cells r@ init_cellvarsrWksV ==--L  ..0 >  # #Hjj.> ?t !rBcSSKJn U"XX#U5nXXl[U[5(aSSKJn [U "U55nUb[U[5(deXhlU$)Nr) FunctionTypepairwise) typesrY__kwdefaults__r9rF itertoolsr[rG__annotations__) rT f_globalsrUdefaultsclosure kwdefaults annotationsrYfuncr[s r@_create_nested_fnrf|s_#  AD$+u%%&8K01   *[$"?"?? ?& KrB>__doc____code__ __closure__ __globals__ __defaults__r]r_c>U=(a [X#5n[R"X5nU[ ;a [ X5nU(a [RRWU5$[R"UW5$![a [ [U5 Nof=fr8) rinspectgetattr_staticAttributeErrorrfn_known_dunder_attrsgetattrr r;r<r.r:)r=fnr?rUsubobjs r@fn_var_getattrrts  0 60F5''1 $$" ,,33FFCC  V ,, 5 !45sA??BBc\\rSrSrSrSrSSjrSSS\S\4S jr S r S r S r g )BaseUserFunctionVariablec6UR5R$r8get_code co_filenameselfs r@ get_filename%BaseUserFunctionVariable.get_filename}}***rBc6UR5R$r8rzco_namer|s r@get_name!BaseUserFunctionVariable.get_name}}&&&rBr=r2returncLURU/UR5QUQU5$r8)inline_user_function_return self_argsr}r=argskwargss r@ call_function&BaseUserFunctionVariable.call_functions, --d4Ndnn6F4N4NPVWWrBrUcSn[UR5U5n[ R RU5$![a US:Xa[U[5(aSnNHf=f)NF__name__T)hasattr get_functionNotImplementedErrorr9NestedUserFunctionVariabler r0r<r}r=rUrHs r@call_obj_hasattr)BaseUserFunctionVariable.call_obj_hasattrsh T..0$7F))0088# z!j7Q&R&R s='A'&A'cp[[R"UR55R5$r8)rDrm signaturer parametersr|s r@inspect_parameter_names0BaseUserFunctionVariable.inspect_parameter_namess'G%%d&7&7&9:EEFFrBc0$r8r}r=s r@ closure_vars%BaseUserFunctionVariable.closure_vars rBrNr=r2rlist[VariableTracker]rdict[str, VariableTracker]rr.) r __module__ __qualname____firstlineno__r~rrstrr.rrr__static_attributes__rrBr@rvrvsd+'X #X&X- X  X 9) 914 9  9GrBrvc^\rSrSrSrSS1\R kr\S5rSSU4Sjjjr U4Sjr S r S r S r S rS rSrS\\\44SjrSSS\4SjrSSS\S\4SjrSU4SjjrSrU=r$)UserFunctionVariablez-Some unsupported user-defined global functionrr is_constantc^[UR[R55 U"XS9$Nr?)r make_guardr CLOSURE_MATCHclsvaluer?s r@create_with_source'UserFunctionVariable.create_with_sources&f'' (B(BCD5((rBrc B>[TU]"S0UD6 [USS5(aSUlOSUl[ U[ R [RR45(dS[U5SU35e[R"USU5nXl g)N_dynamo_marked_constantFTzexpected FunctionType found  _torchdynamo_inliner)super__init__rqrr9r\rYtorchjitScriptFunctionr,rmrnrrr}rrrr __class__s r@rUserFunctionVariable.__init__s "6" 20% 8 8#D $D "u115993K3KLMM *72;-q = M  # #B(=r B&(rBcb>[U[5(a UR$[TU]5$r8)r)rrrras_python_constantr}rs r@r'UserFunctionVariable.as_python_constants( $, - -77Nw)++rBc/$r8rr|s r@rUserFunctionVariable.self_argsrrBcUR$r8rrr|s r@r!UserFunctionVariable.get_function wwrBc.URR$r8)rrrhr|s r@rzUserFunctionVariable.get_codeswwrBc"[R$r8r\rYr|s r@ python_type UserFunctionVariable.python_type!!!rBc4[URSS5SL$N__self__)rqrrr|s r@has_selfUserFunctionVariable.has_selfstww D1==rBc.URR$r8)rrrjr|s r@ get_globals UserFunctionVariable.get_globalssww"""rBcUR(aeURRn[R"[ US9nUR nUR=(d /n[U5VV s/sH*upURcSO[URU5PM, n nn [R"URURUR[![#Xz5V V s/sH upU"XS9PM sn n 5UR$5n UR&(a{UR&Vs0sH(nUURcSO[URUSS9_M* nnUR&R)5VVs0sHunnX"UXS9_M snnU l[*R,"U 5R."U0UD6nUR15 [3UR4R)55n[7UU5 [9UUUR5 UR R$=(d Sn[;U5[;UR RR<5:Xde[#[>R@"5UR RR<U5HunnnURRBnUU;aUUnOUR(ac[E[GURS5U5n[GUS5n[HRJ"UURLU5nURUUUU5nO5[HRJ"UURL5nURUSUU5nUUU'M U$s sn nfs sn n fs snfs snnf![Na [PRR"5nNf=f![Na [PRR"5nN{f=f) z Assume `args` and `kwargs` are VariableTracker arguments for a call to this function, create new bindings for initial locals. )r=N)r>r?T)is_kwrri cell_contents)+rrMroot_tx functoolspartialrArrrk enumerater?r r\rYrhrjrrFziprir]rErmrbindapply_defaultsrG argumentsrKrWlen co_freevarsr^countrNr!rr.r:r ValueErrorr DeletedVariabletrack_cell_existing)r}rSrrrwraprrraidx_defaults_sourcesargr? fake_funcrIkwdefaults_sourcesrJr6rHrbrUcellrNcell_var closure_cellclosure_cell_contents contents_vars r@ bind_argsUserFunctionVariable.bind_argss ####--''  G<!%??(b$H- -KK'D^DKK-M M-  && KK NN KK (+8'F'F S0'F  NN     ** " +A {{*' QdCD + "--335(5DAq4A&8&;<<5(I $ !!),114B6B eoo++-.&)ffbkk2''%%+7|s477#3#3#?#?@@@@" OO tww//;;W OCt"==55L|#'-,t{{M:C  )3<(Q%?#2#8#8 2 24I$L (;; $ ?#2#8#8ASAS#TL(;;D$ U#F4LE H U "(<"?#,#<#<#>L?"?#,#<#<#>L?s<21N *N2/NN"N$!O $OO O+*O+r=r2rUcU[;a[R"X5$[XRUR U5$r8)r$r GetAttrVariablertrrr?r}r=rUs r@ var_getattr UserFunctionVariable.var_getattr\s3 ) ),,T8 8b''4;;==rBcl[URU5n[RR U5$r8)rrrr r0r<rs r@r%UserFunctionVariable.call_obj_hasattras+$'))0088rBcB>UR[RRLa[R "UR5R "U0UD6nURSn[U[5(d)UR5n[SURS35 [U[5(dUR5n[SUS35 URn[R "USS9$UR"(a%[%XRUR5X#5$UR&R(R*(aUR&R(R,(dsSSKJn U biURU R4U R64;aC[RR8R-U5 [:T U]yXU5sSSS5 $[:T U]yXU5$![2a Sn Nf=f!,(df  N0=f) Nrz=`nonstrict_trace` expects a callable, but got value of type <>z) Applying `nonstrict_trace` to function ; however, `nonstrict_trace` currently requires the function to be defined outside `torch.compile` region. Tnonstrict_traceable) FSDPState)rrr_dynamononstrict_tracermrrrr9rvrrrrrr TorchInGraphFunctionVariablerinvoke_and_store_as_constantrMcurrent_tracerunder_activation_checkpoint#allow_side_effects_under_checkpoint/torch.distributed.fsdp._fully_shard._fsdp_stater  Exception _pre_forward _post_forwardrNrr) r}r=rrr6fn_vartypfn_namerrr rs r@r"UserFunctionVariable.call_functiongs 77emm33 3%%dgg.33TDVDEZZ]Ff&>??((*STWT`T`Saabcf&:;; //+))0 2 B99"RVW W   /GGT]]_d  II $ $ @ @II,,PP !U$&&''5*]]//SSTVW 706BXWw$Rv66 !  ! XWs<G>H> H  H  H)rrrFrNr)rrrrrgrv_nonvar_fields classmethodrrrrrrzrrrrGrr.rrrrr __classcell__rs@r@rrs7  " 0 0N ))))",  ">#Uc?6J1KUn>5>S> 9)9149 9 -7 #-7&-7- -7  -7-7rBrcZ^\rSrSrSSU4Sjjjr\S5rSSjrSrU=r $) BuiltinMethodVariableic t>[TU]"S0UD6 [U[R5(deXlgNr)rrr9r\BuiltinMethodTyperrrs r@rBuiltinMethodVariable.__init__s1 "6""e556666rBcURnURnU[L=(a US:H=(d [U5[L=(a US:H$)N__new__ __contains__)rrrFtype frozenset)obj method_self method_names r@is_supported_builtin_method1BuiltinMethodVariable.is_supported_builtin_methodsIll ll u$A )A   * L{n/L rBcURRnURRnUR=(a [ URS5n[ R "XU5nURXX#5$r)rrrrr?rr.r: call_method)r}r=rrr+rU obj_sourceobj_vts r@r#BuiltinMethodVariable.call_functions` gg&& ww[[HZ Z%H  &&r C!!"D99rBrrrr) rrrrr staticmethodr-rrrrs@r@r r sQ    : # :& :- :  : :rBr c ^\rSrSrS\R S\S4U4SjjrSrSr Sr S r S r S rS r \ rS rSrSrSrSrSrSrS\R.4SjrS\\4SjrSrSrSrSSS\SSSSSS 4 U4S!jjr S"r!U=r"$)#LocalGeneratorObjectVariableirT inline_tracerr2c J>[TU]"S0UD6 XlX lX0lgr")rrrTr`r7)r}rTr`r7rrs r@r%LocalGeneratorObjectVariable.__init__s% "6" "*rBcUR$r8)rTr|s r@rz%LocalGeneratorObjectVariable.get_codes yyrBc6UR5R$r8ryr|s r@r~)LocalGeneratorObjectVariable.get_filenamerrBc6UR5R$r8rr|s r@r%LocalGeneratorObjectVariable.get_namerrBc[er8)rr|s r@r)LocalGeneratorObjectVariable.get_functions!!rBcgNFrr|s r@r%LocalGeneratorObjectVariable.has_selfrBc"UR5$r8)rr|s r@r%LocalGeneratorObjectVariable.__name__s}}rBcVURRSUR5S3$)N())rrrr|s r@__str__$LocalGeneratorObjectVariable.__str__s'..))*!DMMO+>rBcSSKJn SSKJnJnJn UR "5nU"U5nU"U5nU"U5n U U U  URU5n U R(dURU5Ul [R"UR5RU5 SSS5 SSS5 SSS5 g!,(df  N=f!,(df  N(=f!,(df  g=f)Nr)"disallow_side_effects_in_generator)r2 save_and_restart_speculation_log(temporarely_allow_writes_to_output_graph)torch._dynamo.side_effectsrNtorch._dynamo.symbolic_convertr2rOrP current_tx_get_inline_tracergenerator_exhaustedforce_unpack_var_sequenceremaining_itemsr ListIteratorVariable reconstruct) r}codegenrNr2rOrPr=savedisallowtemptracers r@rY(LocalGeneratorObjectVariable.reconstructsQ  # - - //35b97; 8T,,R0F--'+'E'Eb'I$  * *4+?+? @ L LW U "8TTTT88TTs;C%CA(C*C2C% C C C" C%% C3c:URRXU5$r8)rrrrs r@r&LocalGeneratorObjectVariable.bind_argssww  622rBcUR$r8r`r|s r@r(LocalGeneratorObjectVariable.get_globals ~~rBc"[R$r8)r\ GeneratorTyper|s r@r(LocalGeneratorObjectVariable.python_types"""rBcpSSKJn URcURX/05UlUR$Nr)InliningInstructionTranslator)rRrkr7build_inline_tracer)r}r=rks r@rT/LocalGeneratorObjectVariable._get_inline_tracers9P    %!>!R!R"b"D !!!rBc~URU5nUR5(a[[U5 [R "[ S[ S05 UR5sSSS5 [ S==[ S-ss'$!,(df  Ol=f![anUeSnAf[a e[aCn[RRRUR55 [ UeSnAff=f[ S==[ S-ss'g![ S==[ S-ss'f=f)Nr inline_call)rT_is_generator_exhaustedr StopIterationr rGr% inline_call_rrrrr  eval_frame skip_coderzr)r}r=r^es r@ next_variable*LocalGeneratorObjectVariable.next_variables((,  ' ' ) ) $]B 7 AH8O&PQ**,RQ _ %-)@ @ %RQ  G%   # MM $ $ . .t}} ? " #R _ %-)@ @ %H_ %-)@ @ %sS$BB, B BBD"B D)B++D>DDD""D<cgrCrrs r@has_unpack_var_sequence4LocalGeneratorObjectVariable.has_unpack_var_sequencerErBrcgNTrrs r@has_force_unpack_var_sequence:LocalGeneratorObjectVariable.has_force_unpack_var_sequencesrBc/nURURU55 M#![a [U5 U$f=fr8)appendrvrr)r}r=rHs r@rV6LocalGeneratorObjectVariable.force_unpack_var_sequencesL  d0045- )"-  s 'AAcURU5nURU5 g![anURU5 SnAgSnAff=fr8)rT_raise_exception_variablerexception_handler)r}r=excr^rus r@_setup_exception-LocalGeneratorObjectVariable._setup_exception%sH((, (  , ,S 1  (  $ $Q ' ' (s% A AA c^URSL=(d URRS:H$)Nr)r7instruction_pointerr|s r@_is_generator_just_started7LocalGeneratorObjectVariable._is_generator_just_started.s*!!T)XT-?-?-S-SWX-XXrBc0[URSS5$)NrUF)rqr7r|s r@rp4LocalGeneratorObjectVariable._is_generator_exhausted1st))+@%HHrBr=rUrrrrr.c>US:XaURU5$US:XaU$US:XaUR5(a7[U5(a'[SU55(d[ [ U5 UR U5nURU5 URU5$US:XaUR U5nUR5(dUR5(aSUl [R"S5$URU[R"[S55 [R S:a7UR"R$S :XaSUl [R"S5$URU5(a[ [(U5 OUS :XaUR U5nURU[U5S :XaUS OUS 5 URU5n[1S[2405nURU[R"US55 URU5 [ [(U5 U$[6TU]qXX45 g![&a SUl [R"S5s$f=f![&a SUl [R"S5s$[*a UR,ceUR,s$f=f![.a ef=f![5U5a U$f=f)N__next____iter__sendc3l# UH*n[U[5=(a URSLv M, g7fr8)r9r0r).0rs r@ ;LocalGeneratorObjectVariable.call_method..Gs0#s$45K#))t:KK#s24closeTr) CALL_INTRINSIC_1throwrr+r__InternalThrowException)rvrrallr TypeErrorrT push_manyrprUr r0rExceptionVariable GeneratorExitsys version_infonext_instructionopnamer RuntimeErrorrsymbolic_resultrr(rrrr0) r}r=rUrrr^retvalexc_typers r@r0(LocalGeneratorObjectVariable.call_method4s : %%b) ) Z K V^..00SYY#-Y;,,R0F   T "%%b) ) W_,,R0F..00D4P4P4R4R-1* 11$77 8%% 33M2F($$///66:LL15F.$55d;; .%%b)),\2>W_,,R0F  %%bSY!^$q'aQ ''+F|6 bIH ?%%b)*E*EhPR*ST""2& )v>M Bd3a) 8-1* 11$77 8) 8-1* 11$77, . --999---  .,%  N1: M  sIA5I#<&J;)K#8K2#'J  J 'K 9$K K # K/2LL)rTr`r7rW)#rrrrr\CodeTyperrrzr~rrrrK__repr__rYrrrrTrvrybuiltinsboolr}rDr.rVrrrprr0rrrs@r@r6r6s +nn + 78 ++'"?HV&3#"A.8==tO/D(YIz4 #z4z4& z4 - z4  z4z4rBr6c\rSrSrSrSrg)4ContextlibContextManagerLocalGeneratorObjectVariableiz .. note:: This is only used when the function is annotated with @contextlib.contextmanager It is a special case of a generator function as we do not allow return a context manager from a torch.compile function. rN)rrrrrgrrrBr@rrsrBrcb^\rSrSrSr\S.S\4U4SjjjrSrSr S Sjr S r U=r $) LocalGeneratorFunctionVariableizlfunctions that behaves like iterators .. note:: This is a wrapper around (Nested)UserFunctionVariable ) generator_clsvtc >>[TU]"S0UD6 XlX lgr")rrrr)r}rrrrs r@r'LocalGeneratorFunctionVariable.__init__s  "6"*rBcXRRR5;a [X5$[URU5$r8)r__dict__keysrqr)r}rUs r@ __getattr__*LocalGeneratorFunctionVariable.__getattr__s7 >>**//1 14& &tww%%rBc6SSKJn URUUUU5$rj)rRrkrl)r}r=rrrks r@_build_inline_tracer3LocalGeneratorFunctionVariable._build_inline_tracers$P,@@       rBc[URR55(deURXU5nURR5nURR 5nUR UUUUR S9$r)rrrzrrrr?)r}r=rrr7rTr`s r@r,LocalGeneratorFunctionVariable.call_function s DGG,,.////11"FC ww!GG'') !!   ;; "  rB)rrr) rrrrrgr6r.rrrrrrrs@r@rrsY3 +  + +&   # & -     rBrc8^\rSrSrSrU4SjrU4SjrSrU=r$)3FunctionDecoratedByContextlibContextManagerVariablei5za .. note:: This is only used when the function is annotated with @contextlib.contextmanager c 4>[TU]"U4S[0UD6 g)Nr)rrr)r}rrrs r@rs%   N  rBc>[TU]XU5n[U[RR R 5(deSUlU$r|)rrr9rr symbolic_convert&InliningGeneratorInstructionTranslatoris_generator_from_ctx_manager)r}r=rrr^rs r@rHFunctionDecoratedByContextlibContextManagerVariable._build_inline_tracerEsR -b?  MM * * Q Q    04, rBr) rrrrrgrrrrrs@r@rr5s    rBrc^\rSrSrSrSU4SjjrS\4SjrSrSr SU4S jjr U4S jr SS S \4U4S jjr Sr U=r$)UserMethodVariableiSz$Some unsupported user-defined methodrc 6>[TU]"SSU0UD6 X lg)Nrrr)rrr*)r}rrr*rrs r@rUserMethodVariable.__init__Vs )B)&)rBchURRSURSURS3$)NrIz, rJ)rrrrr*r|s r@rUserMethodVariable.__repr__Zs...))*!DGG9BtxxjBBrBcUR/$r8r*r|s r@rUserMethodVariable.self_args]szrBc"[R$r8)r\ MethodTyper|s r@rUserMethodVariable.python_type`srBr=r2c>SSKJn U"UR5(aE/UR5QUQn[R "URSS9nUR XU5$URR5(a[UR[R5(a[URSS5nUbURS5(aUS:wdUR(a9URRXRR X#URS 9$Ol["beUR["R$R&La>[R("UR5R XR/UQ7U5$UR(aO[URR*URR 5n[-XUR/5X#5$[0T U]XU5$) Nr )is_nonstrict_trace_callableTrrz torch.nn.ztorch.nn.utils.parametrize)constant) trace_rulesrrrrr r rrMis_root_tracerr9r*NNModuleVariablerq startswithrr0rr1FSDPParamGroupuse_training_stateTorchCtxManagerClassVariablerr rr) r}r=rrr call_argsvar module_attrrrrs r@r UserMethodVariable.call_functioncs > &tww / /2$..*2T2I88TC$$RF; ; 99 # # % %* HHi00+ + "$''<[TU]5SS$Nr+)rrrs r@r*UserMethodVariable.inspect_parameter_namessw.044rBrUc>UR=(a [URU5nUS:Xa UR$US:Xa![R"XR U5$[ TU]X5$)Nr__func__)r?rr*r.r:rrrr)r}r=rUr?rs r@rUserMethodVariable.var_getattrs\>DKK!> : 88O : "((WWf= =w"2,,rBrrr)rrrrrgrrrrrrrrrrrs@r@rrSsm.C#C ;7 #;7&;7- ;7  ;7z5-5-S--rBrcL^\rSrSrSU4SjjrSU4SjjrSrU=r$)WrappedUserMethodVariableic >URSS5 URSS5 [TU]"URUR40UD6 XlX lgNrrr*)rRrrrrr*wrappedcontextr}rrrrs r@r"WrappedUserMethodVariable.__init__sE 4 5$ W[[;F;  rBc>URRU5 [TU] XU5nURR U5 U$r8renterrrexitr}r=rrrHrs r@r'WrappedUserMethodVariable.call_function? 2&r8 " rBrrrrrrrrrrrrrs@r@rr8  # & -    rBrcL^\rSrSrSU4SjjrSU4SjjrSrU=r$)WrappedUserFunctionVariableic >URSS5 URSS5 [TU]"UR40UD6 XlX lgr)rRrrrrrrrs r@r$WrappedUserFunctionVariable.__init__s? 4 5$ .v.  rBc>URRU5 [TU] XU5nURR U5 U$r8rrs r@r)WrappedUserFunctionVariable.call_functionrrBrrrrrs@r@rrrrBrcSnUVs/sH oe"U5PM nnUR5VVs0sH upxXu"U5_M nnnU"U0UD6n URRU U[U5S9$s snfs snnf)Nc[U[R5(aUR5$UR 5$r8)r9r TensorVariableget_real_valuer)xs r@convert-invoke_and_store_as_constant..converts3 a11 2 2##% %##%%rBr)rErMregister_attr_or_moduler) r=rrrUrrrrrIrJress r@r r s& !% %1GAJD %(.  7amF 7 d f C 99 , ,  d# -  & 7s A1A6c^\rSrSrS1\R krSSU4SjjjrSrSrSr Sr Sr S r S r S rS rS rU=r$)rir`c  \>[T U]"S0U D6 [UR5[5(de[UR5[ R 5(de[U[5(deXlX l X0l X@l XPl X`l XplXlgr")rrr9rrr\rrGrrTr`rarcrdrb wrapped_fn) r}rrTr`rarcrdrbr rrs r@r#NestedUserFunctionVariable.__init__s "6"'446<<<<$113U^^DDDD)T****  " $& 5?rBc/$r8rr|s r@r$NestedUserFunctionVariable.self_argsrrBc6URR5$r8)rTrr|s r@rz#NestedUserFunctionVariable.get_codesyy++--rBc"[R$r8rr|s r@r&NestedUserFunctionVariable.python_type rrBcUR(a[e[R"URR 5UR URR 55nUR(aURR 5Ul UR(aURR 5Ul UR(acURR 5n[U[5(aSSKJn [#U"U55n[U["5(deX!lU$)NrrZ)rbrr\rYrTrr`rrarkrcr]rdr9rFr^r[rGr_)r}rerdr[s r@r'NestedUserFunctionVariable.get_functions <<% %!! II ( ( * NN LL + + -  == $ @ @ BD  ??"&//"D"D"FD    **==?K+u--."8K#89 k400 00#.  rBcURSL$r8)rbr|s r@ has_closure&NestedUserFunctionVariable.has_closure&s||4''rBcgrCrr|s r@r#NestedUserFunctionVariable.has_self)rErBcUR$r8rcr|s r@r&NestedUserFunctionVariable.get_globals,rerBc`UR5n[R"UURURR 5UR (a[UR R5OS[S[[UR5R55555nUR(aURR5Ul[R "U5R""U0UD6nUR%5 ['UR(R55n[+UR,R.U5 [1XU5 [3UR5H)upX;deUR4RUn XU 'M+ U$)Nc38# UHn[S5v M g7fr8)r*)rrs r@r7NestedUserFunctionVariable.bind_args..6sS+Ra)D//+Rs)rzr\rYr`rrrarFrErangerrrckeys_as_python_constantr]rmrrrrGrrKrMrrWrrb) r}rSrrrTrer6rHrrUrs r@r$NestedUserFunctionVariable.bind_args/s9}}!!  NN LL + + -*.--E$--%% &T S5T]]_5P5P1Q+RS S   ??"&//"I"I"KD !!$',,d=f= eoo++-...7fd+"4#3#34IC% %%<<%%c*D4L5  rBc&^TRU4Sj5 T"UR5 TRTRUR5/5 T"[ R "URRR55 UR(aT"UR5 O!TRTRS5/5 UR(aT"UR5 O!TRTRS5/5 UR(aT"UR5 O!TRTRS5/5 UR(a=URR5nTRTRU5/5 O!TRTRS5/5 TR[!SS55 UR"(axTRU4Sj5 T"UR"5 TR[!SS55 TR[%S55 TR[!SS55 gg![a T"UR5 Nf=f)Nc0>TR[S5$)Nrf)load_import_fromrrZsr@8NestedUserFunctionVariable.reconstruct..IsG,,X7JKrBFc(>TRSS5$)Nrwrapsr&r'sr@r(r)ms00gFrBr+r T) add_push_nullrT extend_outputcreate_load_const_uncheckedr`r0r<rrracreate_load_constrbrcrdrrrr r)r}rZrds ` r@rY&NestedUserFunctionVariable.reconstructGs K   wBB4>>RST '' (?(?@A == DMM "  ! !7#<#>[TU]"S0UD6 XlX lgr")rrrr5)r}rr5rrs r@rSkipFunctionVariable.__init__|s "6"  rBcUR$r8rr|s r@r'SkipFunctionVariable.as_python_constants zzrBc~[U5(d([UR[R55 U"XS9$r)r(rrrFUNCTION_MATCHrs r@r'SkipFunctionVariable.create_with_sources3.u55 &++L,G,GH I5((rBr=r2c `[R"URSS5(a/[S[ UR5SURS3S/S9 gUR[ R RLaAURSS5nU(aUR5n[S S US US 3S U3S/S9 g[UR[R5(aESURS3n[ R RRU5 [U5 g[!URSS5n[R""UR5nSUSUS3nSUS3/n SU;aU SUSUS3S/- n US+:XaS,n/n UR,(a UR,OS-n [S.S/URR&S0US1U 3UU S9 g![$GaF SS1n URR&U ;a#SURR&S US3nS!S"/n NURR&bURR&R)S#5(aaS$URR&S US 3nS%/n [ R RRUS&-S&R+U 5-5 GNAS'URR&S US(3nS)S*/n [ R RRUS&-S&R+U 5-5 GNf=f)2N_torchdynamo_disableFz1Skip calling `torch.compiler.disable()`d functionzSkip calling function `z4` since it was wrapped with `torch.compiler.disable`z(Remove the `torch.compiler.disable` call)gb_typer explanationhintsmsgz%Call to `torch._dynamo.graph_break()`z0Called `torch._dynamo.graph_break()` with args `z `, kwargs ``z$User-inserted graph break. Message: z.Remove the `torch._dynamo.graph_break()` call.z2Graph break due to unsupported wrapper descriptor zM. Please file an issue on GitHub so the PyTorch team can add support for it. rzz?Dynamo developers have intentionally marked that the function `z ` in file `z` should not be traced.zAvoid calling the function `z`.r zRemove the function `z` or the file `zv` from torch/_dynamo/trace_rules.py. More graph breaks may occur as a result of attempting to trace into the function.z Please file an issue to PyTorch._abc _warningsz6Dynamo does not know how to trace the Python builtin `.zIf you are attempting to call a logging function (e.g. `_warnings.warn`), you can try adding it to `torch._dynamo.config.reorderable_logging_functions`.zKPlease file an issue on GitHub so the PyTorch team can add support for it. optreez*Dynamo cannot trace optree C/C++ function zi Consider using torch.utils._pytree - https://github.com/pytorch/pytorch/blob/main/torch/utils/_pytree.py z/Dynamo does not know how to trace the builtin `z.` This function is either a Python builtin (e.g. _warnings.warn) or a third-party C/C++ Python extension (perhaps created with pybind).zIf it is a Python builtin, please file an issue on GitHub so the PyTorch team can add support for it and see the next case for a workaround.aIf it is a third-party C/C++ Python extension, please either wrap it into a PyTorch-understood custom operator (see https://pytorch.org/tutorials/advanced/custom_ops_landing_page.html for more details) or, if it is traceable, use `torch.compiler.allow_in_graph`.allow_in_graphzFound an allow_in_graph decorator to a function which is created inside the parent function that is getting compiled. This is not supported for now.zz,Attempted to call function marked as skippedzmodule: z , qualname: z, skip reason: )rmrnrrrrr  graph_breakgetrr9r\WrapperDescriptorTypeutils warn_oncerrqgetfilerrrjoinr5) r}r=rrgraph_break_msgrCqualnamepathrArBknown_python_builtin_modulesr5s r@r"SkipFunctionVariable.call_functionsv  ! !$**.De L L KDJJ5djj\Auv>  ZZ5==44 4$jj5O"1"D"D"F ?J4&P[\b[ccdeB?BSTD   E$?$? @ @DTZZLQ?@  MM   ) )# . # tzz>;OPH< Ytzz2UV^U_` $v%<> 38*B? D(/zOAA; E`++? $(KKT[[5GF F"4::#8#8"9hZ_e^fg'  c( Y06 /D,::((,HH JJ112!H:RA iGEJJ))5 --88BB$NtzzOdOdNeefgofppq"rK^E MM''11+2DtyyQVGW2WXJ$**J_J_I``abjaklab m;EMM''11+2DtyyQVGW2WXQ( Ys"AGA L-(BL- AL-,L-ch[RR[URU55$r8)r r0r<rrrs r@r%SkipFunctionVariable.call_obj_hasattrs$))00T1JKKrBrUcU[;a[R"X5$[XRUR U5$r8)r$r rrtrr?rs r@r SkipFunctionVariable.var_getattrs3 ) ),,T8 8b**dkk4@@rB)r5rr8rrr=r2)rrrrr.rrrrrrrrrrrrs@r@r4r4us  ' 'N  ))o #o&o- o  obLA5ASAArBr4cZ^\rSrSrSrSU4SjjrSU4SjjrS SjrSrU=r $) WrapperUserFunctionVariableia Used to represent a wrapper object that contains the actual callable as an attribute. For example, torch.jit.script/trace have the original function at their _torchdynamo_inline attribute. Similarly, functions with __script_if_tracing_wrapper have the original attr at "__original_fn". c >>[TU]"S0UD6 XlX lgr")rr wrapper_obj attr_to_trace)r}r_r`rrs r@r$WrapperUserFunctionVariable.__init__s "6"&*rBc>X R:Xa`[URUR5nUR=(a [ URU5n[ R "XU5$[TU]!X5$r8) r`rqr_r?rr.r:rr)r}r=rUr>r?rs r@r'WrapperUserFunctionVariable.var_getattrsc %% %$**D,>,>?C[[BZ T%BF"((&9 9w"2,,rBc[R"[R5R X[R "UR 5/UQU5$r8)r rr getattr_and_tracerr0r`rs r@r)WrapperUserFunctionVariable.call_functionsJ --  ' ' - y11$2D2DEMMv  rB)r`r_rr[r) rrrrrgrrrrrrs@r@r]r]sB+ -  #  &  -     rBr]c\[RR5(aSSKJn U$0$)Nrtraceable_collective_remaps)r distributed is_available)torch.distributed._functional_collectivesrirhs r@_traceable_collective_remapsrm*s) %%'' +* IrBc[RR5(dS5eU[5R 5;deUR nUR S5n[X25$)NzIllegal invocation.z)torch.distributed._functional_collectives)rrjrkrmvaluesr import_sourcer)r=rr inner_name path_sources r@_traceable_collectives_sourcers5sc    ) ) + +B-BB + -/668 88 8J""#NOK k ..rBc^\rSrSrSrS U4Sjjr\S Sj5r\S5r\S Sj5r S Sjr Sr U=r $) !CollectiveFunctionRewriteVariablei>a Some of the torch.distributed.* collective APIs are possible to rewrite to 'traceable' collectives. This class provides both a way to check if a function is remappable, and perform the remapping. In the case that a function is 'remappable' but only for some combinations of call-time arguments, we check the args at `call_function` time and fall back to graph-breaking if needed. This is no worse than status-quo as we currently graph-break on all distributed.* collectives. c b>[TU]"U40UD6 [U[5(deX lgr8)rrr9rreplacement_var)r}rrrwrrs r@r*CollectiveFunctionRewriteVariable.__init__Is/ &v&/+?@@@@.rBc f[RX5upE[U4[U4SU0UD6US.UD6$)Nr?)rwr?)rurewriter)r=old_fnr?optionsnew_fn new_sources r@r<(CollectiveFunctionRewriteVariable.createNsI>FFrR0  0V VgV    rBcV[R"U5=(a U[5;$r8)rm isfunctionrm)variables r@ can_rewrite-CollectiveFunctionRewriteVariable.can_rewriteXs%   x ( WX9U9W-W rBc6[5UnU[X54$r8)rmrs)r=rrr}s r@rz)CollectiveFunctionRewriteVariable.rewrite^s -/34R@@@rBcSSKJn SSKJn [R "UR 5n[UR"U0UD6R5nSnSU;a0USR5(a[SUR 35 UR URURUR4;apURS5nUb UR OUR"SR$nX;a['SU35e[(R*R-XX5US'UR.R1XU5$)Nr)REDUCE_OP_TO_STRrasync_opzBCollectiveFunctionRewriteVariable can't support async_op=True for opzUnsupported all_reduce op: )torch.distributedrjrlrrmrrrrGrrrr all_reducereduce_scatter_tensor_reduce_scatter_baserLrrdefaultrr r0r<rwr) r}r=rrdistrr reduce_op_var reduce_ops r@r/CollectiveFunctionRewriteVariable.call_functioncs6 )N%%dgg. innd5f5??@  F:$6$I$I$K$K TUYU\U\T]^  77 OO  & &  % %  #JJt,M!,##))$/77  0 #>yk!JKK$55<< +F4L##11"FCCrB)rwrr[r) rrrrrgrr4r<rrzrrrrs@r@ruru>s/     AA'D #'D&'D- 'D  'D'DrBruc<^\rSrSrSU4SjjrSrU=r$)FunctoolsWrapsVariableic>^U(d+[T5S:XaU4Sjn[R"U5$[TU]UTU5$)Nr+c>[U[R5(aURTSS9$[ SUS35 g)Nr)r zfunctools.wraps(rJ)r9r rcloner)rrrs r@r,3FunctoolsWrapsVariable.call_function..wrapss=b)"F"FGG88tAw877 0A67rB)rr LambdaVariablerr)r}r=rrr,rs ` r@r$FunctoolsWrapsVariable.call_functions> #d)q. 8 ++E2 2w$Rv66rBrr)rrrrrrrrs@r@rrs37 #7&7- 7  77rBrc4\rSrSrSrSSjrSrg)CollectionsNamedTupleFunctionicUR$r8rr|s r@r0CollectionsNamedTupleFunction.as_python_constantrrBc P[X#5nU(a~UR"UVs/sHoUR5PM sn0UR5VVs0sHupgXgR5_M snnD6n[R "U[ 5S9$[S5 gs snfs snnf)N) mutation_typez!namedtuple with non constant args)r"rrrrEr UserDefinedClassVariabler-r) r}r=rr constant_argsrrIrJrs r@r+CollectionsNamedTupleFunction.call_functions ,D9 GG267$Q&&($79?H1**,,HE55%5%7  9: 8Hs BB"rNr)rrrrrrrrrBr@rrs2; #;&;- ;  ;rBrc^\rSrSrS\SS4U4SjjrSrSrSrSS jr S S S \ S\4S jr S S S \ 4Sjr Sr SrSrU=r$)FunctoolsPartialVariableirerNc >[TU]"S0UD6 Xl[U[5(deX l[U[ 5(deX0l[R"[5Ul gr") rrrer9rDrrGkeywordsrrr& fake_value)r}rerrrrs r@r!FunctoolsPartialVariable.__init__s\ "6" $%%%% (D)))) $++H5rBc"[R$r8)rrr|s r@r$FunctoolsPartialVariable.python_types   rBcR^TRU4Sj5 T"UR5 UR(aTRUR5 UR(d2TR [ [UR5S-S55 gTRURR55 [URR55nTR TR[U5[UR5-S-US55 g)Nc(>TRSS5$)Nrrr-r'sr@r(6FunctoolsPartialVariable.reconstruct..sg&>&>{I&VrBr+F) r.rerforeachrr/rrrorFrcreate_call_function_kw)r}rZrs ` r@rY$FunctoolsPartialVariable.reconstructsVW  99 OODII &}}  ! !"6s499~7I5"Q R  ,,./T]]'')*  + +CIDII,F,JDRW X rBc"UR5$r8rr|s r@r%FunctoolsPartialVariable.get_function&&((rBr=r2cxURU-n0UREUEnURRXU5$r8)rrrer)r}r=rr merged_args merged_kwargss r@r&FunctoolsPartialVariable.call_functions< ii$& 34==3F3 yy&&r FFrBrUc[RR[[R "[ 5U55$r8)r r0r<rrrr&rs r@r)FunctoolsPartialVariable.call_obj_hasattrs2))00 I%%h/ 6  rBcUR=(a [URU5nUS:Xa UR$US:Xa[R"UR US9$US:XaWUR R5VVs0sHupE[R"U5U_M nnn[R"XcS9$[[U5 gs snnf)Nrerrr) r?rrer ListVariablerrrEr0r<ConstDictVariablerro)r}r=rUr?rIrJrEs r@r$FunctoolsPartialVariable.var_getattrs>DKK!> 6>99  6>))$))FC C : ?C}}?R?R?TU?Ttq%,,Q/2?TEU..uD D 4Vs#Cc B[R"URR5/URVs/sHoR5PM snQ70UR R 5VVs0sHup#X#R5_M snnD6$s snfs snnfr8)rrrerrrrE)r}rrIrJs r@r+FunctoolsPartialVariable.as_python_constants  II ( ( * 26)) <)3$$&) < 6:]]5H5H5JK5JTQq&&((5JK  <K B 4Bc B[R"URR5/URVs/sHoR5PM snQ70UR R 5VVs0sHup!X!R5_M snnD6$s snfs snnf)zcSimilar to as_python_constant(), but add ID_MATCH guards to try to force things to become constants)rrreguard_as_python_constantrrrE)r}rJrIs r@r1FunctoolsPartialVariable.guard_as_python_constants  II . . 0 48II >Iq((*I > <@==;N;N;PQ;P41q,,..;PQ  >Qr)rrrerr)rrrrr.rrrYrrrrrrrrrrs@r@rrs 6_ 64 6!  )G #G&G- G  G ) 14   55 5S 5   rBrc*^\rSrSrSSS1\R kr\\R"S5S\ \ S\ 4\ R44Sj55r\S 5rS\SS4U4S jjr\S\4S j5rS rS rSSjrSSjrSrSrU=r$)PolyfilledFunctionVariableirrr  traceable_fnNr.c0$r8r)rs r@_get_polyfill_handlers1PolyfilledFunctionVariable._get_polyfill_handlers s  rBc^[UR[R55 U"XS9$r)rrrr<rs r@r-PolyfilledFunctionVariable.create_with_sources&f'' (C(CDE5((rBc D>[TU]"S0UD6 XlUR5R X5n[ U5(d SUSU35eSH,n[ X4S5nU(dM[ U5(deUn O [SUS35eX0lX`l g)NzPolyfill handler z is not callable for )__torch_dynamo_polyfill____python_implementation__z# does not have a traceable functionr) rrrrrrLcallablerqrr r)r}rrrhandlercandidate_attr candidaterrs r@r#PolyfilledFunctionVariable.__init__s "6"--/33B;  X$5gY>STVSW"XX  N >Iy ****(  #G9,OP & ,rBcUR$r8)rr|s r@ polyfill_fn&PolyfilledFunctionVariable.polyfill_fn/s   rBc0[URSS5$)N*__torch_dynamo_can_constant_fold_through__F)rqr r|s r@can_constant_fold_through4PolyfilledFunctionVariable.can_constant_fold_through3s OOI5  rBc"UR5$r8rr|s r@r'PolyfilledFunctionVariable.get_function8rrBc rUR5(a[X#5(awUR"UVs/sHoDR5PM sn0UR 5VVs0sHupVXVR5_M snnD6n[ R "X5$UR[RLGa<[U5S:XGa,U(Gd$[US[R[R45(a[SUSR55(a[RR!UUR"R%S[&R([+SUSR55405[&R("USRVs/sH:n[U[R,5(a UR.O UR0PM< sn5S9$[ R "XR25nUR5XU5$s snfs snnfs snf)Nr+rc3# UH~n[U[R5=(a [UR[5=(d7 [U[R 5=(a UR 5[Lv M g7fr8)r9r r0rintSymNodeVariabler)rrs r@r;PolyfilledFunctionVariable.call_function..Rsd'AAy99:Wz!''SV?WYq)";";<WTWAWY&sBBrc3@# UHoR5v M g7fr8)as_proxy)ras r@rr]s?A::<<s)sym_num)rr#rrrrEr.r:rsumrr9r r TupleVariablerrr<rM create_proxyrsym_sumrFr0rrrr) r}r=rrrrIrJrHtraceable_function_variables r@r(PolyfilledFunctionVariable.call_function;s  ) ) + +0M 1 1 6:;d**,d;=C\\^L^TQq..00^L  #((4 4 GGx|| #D Q47Y%;%;Y=T=T$UVVa ,,33 &&#MM?a??A   "&a  "/A *!Y-G-GHHGG!"+"/  4 ('6&;&;B@Q@Q&R#*886JJO<L4sH)%H.-AH4cvUS:XaURXU5$[URUS5nUcSUSUR35e[U5(dSUSUR35e0nUR(a[ URU5US'[ U40UD6nURXU5$)N__call__zMember z not found in z is not callable in r?)rrqrrr'r?rr)r}r=rUrrmethodr|polyfilled_method_variables r@r0&PolyfilledFunctionVariable.call_methodos : %%b7 7$-!JWTF. #JJ!6""QgdV3Gy$QQ" ;; *4;; =GH %?%R'%R")77&IIrBcUR$r8rr|s r@r-PolyfilledFunctionVariable.as_python_constantrrB)rrrr rrrrrrr.)rrrrr.rrr lru_cacherGrrr\rYrrr5rpropertyrrrrr0rrrrs@r@rrs   ' ' NtHS#X,>@R@R,R'S)) -2-D-.!R!! )2K #2K&2K- 2K  2KhJ& J - J  J&rBrc\rSrSrSrSrg)TracebackVariableicgr8rrs r@rTracebackVariable.call_functionssrBrN)rrrrrrrrBr@rrs2rBrc:^\rSrSrU4SjrSrSrSrSrU=r $)SysFunctionVariableic 2>[TU]"S0UD6 Xlgr")rrr)r}rrrs r@rSysFunctionVariable.__init__s "6" rBcz[UR5(aKURSnURnSn[R"X5U[R"X5/nOA[ R "S5[ R "S5[ R "S5/n[ R"U5$)N)r exn_vt_stackrr.r:r r0r)r}r=exnrtbrEs r@exc_infoSysFunctionVariable.exc_infos r  //"%C,,CB%%b.%%b-E**40**40**40E &&u--rBc>URU5RS$r)rrErs r@ exceptionSysFunctionVariable.exceptions}}R &&q))rBcUR[RLaURU5$UR[RLaURU5$[ SURR 35 g)Nzsys.)rrrr rrrs r@r!SysFunctionVariable.call_functionsZ :: %==$ $ ZZ3== (>>"% % D!4!4 56 7rBr9) rrrrrrr rrrrs@r@rrs.$*88rBr)TMADescriptorMetadataTritonHOPifierc\rSrSrS\S\4SjrS\S\4Sjr S\S\4Sjr S\ \ RRRS 44S jrS rS rS rSrS\S\4SjrSSS\\SS4SjrS\4SjrSrg)DynamoTritonHOPifierirCrc[U5er8)r)r}rCs r@raise_unsupported&DynamoTritonHOPifier.raise_unsupporteds #rBmaybe_callablec.[U[[45$r8)r9rr)r}rs r@ is_callable DynamoTritonHOPifier.is_callables 79MN  rBr>cUR$r8r9)r}r>s r@ get_valueDynamoTritonHOPifier.get_values yyrB.c~SSKJn [X5(aUR5$[ S[ U535 g)Nr+)BaseListVariablezgrid for the triton kernel is )listsrr9rrr()r}gridrs r@ check_gridDynamoTritonHOPifier.check_grids0+ d - -==? " :4:,G HrBcUR5VVs0sH%upE[RRU5U_M' nnnUR X2/05nU$s snnfr8)rEr r0r<r)r}rmetar=rIrJs r@ call_gridDynamoTritonHOPifier.call_gridsSDHJJLQLDA **11!4a7LQ!!"fb1 Rs,AcXSSKJn URXA5nURXBU5nU$)Nr+)SourcelessBuilder)builderr&r<r) r}user_fnrrr=rr&wrapped_user_functionrHs r@call_user_defined_fn)DynamoTritonHOPifier.call_user_defined_fns-. 1 8 8 E&44RvF rBclSSKJn U"U[URU55R U5nU$)Nr+)VariableBuilder)r'r-r kernel_source_wrap)r}user_objr=rrUr-wrapped_user_objs r@wrap_user_defined_obj*DynamoTritonHOPifier.wrap_user_defined_objs7,*  811dV= %/  rBcpURU5nUVs/sHo3R5PM nnU$s snfr8)unpack_var_sequencer)r}configsr=configs r@maybe_unpack_configs)DynamoTritonHOPifier.maybe_unpack_configss<--b1DKK72247KLs3rHcnUR5(dURS5 UR5$)Nz`@triton.heuristics must return constant values because configs can only contain constant values.)is_python_constantrr)r}rHs r@maybe_unpack_heuristic_result2DynamoTritonHOPifier.maybe_unpack_heuristic_results3((**  " "r ..00rBrTritonKernelVariablercURc[U5S:waURS5 [U5"URUR USUR S9$)Nr+z7Triton kernels should be called with only a single gridr)kernel kernel_idxrr.)rrrr(r@rAr?)r}rrs r@ call_getitem!DynamoTritonHOPifier.call_getitemsZ == $D Q  " "I H~??**a"//   rBc `SSKJn SSKJn 0n[ UR 55HHnX8n [ U [5(dMU R5Xx'U RRX8'MJ UR5VV s0sH%up[RRU5U _M' n nn SSKJn Jn UR5VV s0sH'up[ X5(dMXR#5_M) n nn U R5VV s0sHup[ X5(aMX_M nnn UR%5Hbn U R'5n [ U [R([R*45(aMDUR-S[/U 5S35 Md U R1U 5nU"U[25nUR4R7SU S UR8UUUUR;5S .5 [R"S5$s sn nfs sn nfs sn nf) Nr+r/)rr)kernel_side_tabletriton_kernel_wrapper_mutationz.Unexpected argument type for a Triton kernel: rGrr)rAconstant_args_idxrtma_descriptor_metadatar)rr0dictsrrDrr9TMADescriptorVariable to_metadatadata_ptr from_tensorrEr r<*torch._higher_order_ops.triton_kernel_wraprErFrrorealizerrrrepradd_constant_argsrGrMrrAr)r}rgridscombined_args_rawr=r0rrHrIrJ combined_argsrErFrnon_constant_argsrGr"s r@call_HOPDynamoTritonHOPifier.call_HOPs.,:<',,./A!$A!233-.]]_'*'(zz'='=!$ 0*//1 1  & & - -a 0! 31   *//1 1!. &A##% %1  &++- -a2 AD-  #))+A Aa)":":IiHr3rr4r@rArr.rc >URSS5Ul[TU] "S0UD6 [R XX#5 g)Nr.r)rRr.rr dynamo_triton_hopifier_singleton init_variable)r}r@rArrrs r@rTritonKernelVariable.__init__Ns6#ZZ> "6"(66tZVrBc.[RXX15$r8)r]call_triton_kernelrs r@r"TritonKernelVariable.call_functionSs 0BB   rBc>US:Xa[RX5$US:Xa[RXXA5$[TU]XX45$)N __getitem__run)r]rBcall_runrr0)r}r=rUrrrs r@r0 TritonKernelVariable.call_method]sL = 3@@L L U]3<? ? rB)r.rrr)rrrrr_rrrrr0rrkrrrs@r@r>r>Hs   W  # & -     ;& ; - ;  ;ssrBr>cH^\rSrSrSU4SjjrSrSrSrU=r$)rJivc >[U[R5(de[TU]"S0UD6 XlX lX0lX@lgr") r9r DataPtrVariablerrrLdims block_dims element_size)r}rLrprqrrrrs r@rTMADescriptorVariable.__init__wsC(I$=$=>>>> "6"  $(rBcURVs/sHoR5PM snURVs/sHoR5PM snURR54$s snfs snfr8)rprrqrr)r}dims r@rK!TMADescriptorVariable.to_metadatasX'+yy 1y\\^y 1'+ 7\\^ 7    & & (  1 7s A.A3c^^TRUU4Sj5 TRRT5 /TRQTRQTR PnTR U5 TR[U5S-S5 g)NcV>TRSS[TR5S35$)Nz$triton.tools.experimental_descriptorcreate_d_tma_descriptor)r&rrp)rZr}sr@r(3TMADescriptorVariable.reconstruct..s)G,,6#dii.))9:rBr+F) r.rLrYrprqrrrrr)r}rZrs`` r@rY!TMADescriptorVariable.reconstructsp  !!'*@@T__@d.?.?@c$i!mU3rB)rqrLrprr)rLzvariables.DataPtrVariablerplist[ConstantVariable]rqr}rrr0) rrrrrrKrYrrrs@r@rJrJvs= )- )' )- ) ) )  4 4rBrJcN^\rSrSrS\SS4U4SjjrSSjrSrU=r$) CreateTMADescriptorVariableirankrNc B>US;de[TU]"S0UD6 Xlg)N)r+r r)rrr)r}rrrs r@r$CreateTMADescriptorVariable.__init__s& v~~ "6" rBc$SU;aUSOUSn[U[R5(d[SURS35eURS:Xa@[ U5[ U5-S:XdeSU;aUSOUS/nSU;aUSOUS /nO][ U5[ U5-S :XdeS U;aUS OUSS U;aUS OUS /nS U;aUS OUSSU;aUSOUS/nSU;aUSOUSn[ UUUUS9$)Nptrrz8Please ensure there were no graph breaks between create_z3d_tma_descriptor and the upstream .data_ptr() call.r+ru block_dimr dim1dim0 block_dim1r block_dim0rrr)rLrprqrr)r9r rorrrrJ)r}r=rrrrprqrrs r@r)CreateTMADescriptorVariable.call_functionsd %fUmDG#y8899))%$$  99>t9s6{*a/ //!&&u d1gD(3f'<{#$q'Jt9s6{*a/ //"(F"2vQ"(F"2vQD )5(>|$DG(4(>|$DGJ2@61Ivn-tTVx $!%   rB)rr) rrrrrrrrrrs@r@rrsJ  ( #( &( - (  ( ( rBrr8r[)nrgrrrmr^rr\collections.abcrtypingrrrrrtyping_extensionsr unittest.mockr rrr r bytecode_transformationrrrrrrrrrrrrrrrguardsrrr?rrr r!rNr"r#r$r%r&r'r(r)r*baser,r-r.rr0#torch.distributed.fsdp._fully_shardr1ModuleNotFoundErrorrRr2rNr3r4r5rArKrGrrWrfrprtrvrr r6rrrrrrr rr4r]rmrsrurrrrrrr rrr]r>rJrrrBr@rsR. $BB# #VV    1NN   =<&E D  T" A. $sO';"< "(-"DF73F7R:4:N b$8#9 +?+\!4O!4H2 /2 c:s&HHH