a h @sddlZddlZddlZddlZddlmZddlmZmZddl m Z m Z m Z m Z mZddlZddlmZmZmZmZmZmZmZmZmZmZddlmZe rddlmZdd lm Z dd l!m"Z"m#Z#gd Z$e%e&Z'Gd d d eZ(ej)GdddZ*GdddZ+e+Z,Gddde+Z-Gddde+Z.ddde/e e0e e0e1e+dfdddZ2ej)GdddZ3ej)Gddde3Z4ej)Gdd d Z5ej)Gd!d"d"e3Z6ej)Gd#d$d$e3Z7ee4e6e7fZ8ej)Gd%d&d&Z9e8e e0e0ge:d'fd(e;e/e1e0e0ffe:e1d)e:e1d'ed*e fe;e/d+fedIe;e/e fd4d5d6Z?Gd7d8d8Z@Gd9d:d:ZAd;d<ZBe;e/e fee;e/e fe1e e:e dfd=d>d?ZCe;e/e fee;e/e fe1e e:e dfe:e8d@dAdBZDee;e/e fe1e e:e dfdCdDdEZEe/ee;e/e fe1e e:e fee;e/e fe1e e:e fdFdGdHZFdS)JN) defaultdict)autoEnum)AnyCallableOptional TYPE_CHECKINGUnion) _get_node_type BUILTIN_TYPESkeystrLeafSpec MappingKey SequenceKeySUPPORTED_NODES tree_flattentree_maptree_map_with_path)ExportedProgram)Symbol)Source)ShapeEnvStrictMinMaxConstraint) ConstraintDimdims*refine_dynamic_shapes_from_suggested_fixesAdditionalInputsc@s"eZdZdZeZeZeZdS) _DimHintTypez Enum for dynamic shape hints. - AUTO means automatic inference of shape (static or dynamic). - STATIC means static shape (always specialized). - DYNAMIC means dynamic, will error out if specialized. N)__name__ __module__ __qualname____doc__rAUTOSTATICDYNAMICr'r'I/var/www/auris/lib/python3.9/site-packages/torch/export/dynamic_shapes.pyr-src@szeZdZUeed<dZeeed<dZeeed<dZ ee ed<e ddZ e d d Z e d d Zddd ddZdS)_DimHinttypeNminmaxT_factorycCs ttjSN)r)rr$r'r'r'r(r$Asz _DimHint.AUTOcCs ttjSr.)r)rr&r'r'r'r(r&Esz_DimHint.DYNAMICcCs ttjSr.)r)rr%r'r'r'r(r%Isz_DimHint.STATICreturncCs||jstdt|d|dus2|dks2Jd|dusJ|dksJJd|dusj|dusj||ksjJdt|j||ddS) N'z' object is not callablerzmin must be non-negativezmax must be non-negativezmin must be <= maxF)r+r,r-)r- TypeErrorr*r))selfr+r,r'r'r(__call__Ms  z_DimHint.__call__)NN)r r!r"r__annotations__r+rintr,r-bool staticmethodr$r&r%r4r'r'r'r(r):s    r)c@seZdZdZeZeZeZdddee e e e dddZ dddd Z ddd d Z ddd d ZddddZddddZddddZedddZddddZeee e edddZddZdS)ra The `Dim` class allows users to specify dynamism in their exported programs. By marking a dimension with a `Dim`, the compiler associates the dimension with a symbolic integer containing a dynamic range. The API can be used in 2 ways: Dim hints (i.e. automatic dynamic shapes: `Dim.AUTO`, `Dim.DYNAMIC`, `Dim.STATIC`), or named Dims (i.e. `Dim("name", min=1, max=2)`). Dim hints provide the lowest barrier to exportability, with the user only needing to specify if a dimension if dynamic, static, or left for the compiler to decide (`Dim.AUTO`). The export process will automatically infer the remaining constraints on min/max ranges and relationships between dimensions. Example:: class Foo(nn.Module): def forward(self, x, y): assert x.shape[0] == 4 assert y.shape[0] >= 16 return x @ y x = torch.randn(4, 8) y = torch.randn(8, 16) dynamic_shapes = { "x": {0: Dim.AUTO, 1: Dim.AUTO}, "y": {0: Dim.AUTO, 1: Dim.AUTO}, } ep = torch.export(Foo(), (x, y), dynamic_shapes=dynamic_shapes) Here, export would raise an exception if we replaced all uses of `Dim.AUTO` with `Dim.DYNAMIC`, as x.shape[0] is constrained to be static by the model. More complex relations between dimensions may also be codegened as runtime assertion nodes by the compiler, e.g. (x.shape[0] + y.shape[1]) % 4 == 0, to be raised if runtime inputs do not satisfy such constraints. You may also specify min-max bounds for Dim hints, e.g. `Dim.AUTO(min=16, max=32)`, `Dim.DYNAMIC(max=64)`, with the compiler inferring the remaining constraints within the ranges. An exception will be raised if the valid range is entirely outside the user-specified range. Named Dims provide a stricter way of specifying dynamism, where exceptions are raised if the compiler infers constraints that do not match the user specification. For example, exporting the previous model, the user would need the following `dynamic_shapes` argument:: s0 = Dim("s0") s1 = Dim("s1", min=16) dynamic_shapes = { "x": {0: 4, 1: s0}, "y": {0: s0, 1: s1}, } ep = torch.export(Foo(), (x, y), dynamic_shapes=dynamic_shapes) Named Dims also allow specification of relationships between dimensions, up to univariate linear relations. For example, the following indicates one dimension is a multiple of another plus 4:: s0 = Dim("s0") s1 = 3 * s0 + 4 Nr+r,)namer+r,cCstddlm}|durdn|}|dur(|n|}||ksHJd|d||s^Jd|||_||_||_dS)Nrint_ooz(Cannot create Dim with inconsistent min=, max=z)Dim name must be a valid identifier, got )torch.utils._sympy.numbersr< isidentifierr r+r,)r3r:r+r,r<_min_maxr'r'r(__init__s z Dim.__init__r/cs6ttur$tdd|jd|fddS)NzAttempted to add z to m, where an integer was expected. (Only increasing linear operations with integer coefficients are supported.)cs|Sr.r'xotherr'r(zDim.__add__..r*r6NotImplementedErrorr _deriver3rGr'rFr(__add__s  z Dim.__add__cCs||Sr.r'rMr'r'r(__radd__sz Dim.__radd__cs6ttur$tdd|jd|fddS)NzAttempted to subtract z from rCcs|Sr.r'rDrFr'r(rHrIzDim.__sub__..rJrMr'rFr(__sub__s  z Dim.__sub__cCstd|jddS)NzAttempted to negate zN. (Only increasing linear operations with integer coefficients are supported.))rKr rMr'r'r(__rsub__s z Dim.__rsub__cs>ttusdkr,tdd|jd|fddS)NrzAttempted to multiply z with zu, where a positive integer was expected. (Only increasing linear operations with integer coefficients are supported.)cs|Sr.r'rDrFr'r(rHrIzDim.__mul__..rJrMr'rFr(__mul__s z Dim.__mul__cCs||Sr.r'rMr'r'r(__rmul__sz Dim.__rmul__cCsddlm}t|||jS)Nr)sympify)sympyrTstrr )r3fnrTr'r'r( _derived_names zDim._derived_namecCst||||Sr.) _DerivedDimrXr3rWr'r'r(rLsz Dim._derive)r:min_max_r0cCsddlm}|dkrd}||kr$d}|dur@|dur@d|dS|durZd|d|dS|durtd|d|dSd|d|d |dS) Nrr;zDim('z')z', max=)z', min=r=)r>r<)r:r[r\r<r'r'r( _readables  z Dim._readablecCst|j|j|jSr.)rr_r r+r,r3r'r'r(__repr__sz Dim.__repr__)r r!r"r#r)r$r&r%rVrr6rBrNrOrPrQrRrSrXrLr8r_rar'r'r'r(rVs&:    rc@s6eZdZdZedddZeddZeddZd S) _StaticDimz Class for static :func:`Dim` types. This class is only for setting and checking static dim constraints, and the user should never interact with it. valuecCst||_||_dSr.)rVr rd)r3rdr'r'r(rBs z_StaticDim.__init__cCs|jSr.rcr`r'r'r(r+sz_StaticDim.mincCs|jSr.rcr`r'r'r(r,sz_StaticDim.maxN) r r!r"r#r6rBpropertyr+r,r'r'r'r(rbs  rbc@sJeZdZdZeeedddZeddZ eddZ d d Z d d Z d S)rYa Class for derived :func:`Dim` types. Currently we only support increasing linear expressions with integer coefficients. In other words, a derived Dim can always be written in the form Ax + B, where x is a regular Dim (i.e., non-derived Dim), A and B are integers, and A is positive. (In particular, the latter ensures that x < y => Ax + B < Ay + B.) These restrictions on the form of derived Dims makes the metatheory simpler: e.g., it simplifies computing ranges for derived Dims, solving for underlying regular Dims, deciding equalities between derived Dims, and so on. The function lambda x: Ax + B is expressed by `fn`, where x is a normal Dim, `root`. The range of a derived Dim is computed by mapping `fn` over the range of its `root`. )r:rootrWcCs||_||_||_dSr.)r rfrW)r3r:rfrWr'r'r(rB sz_DerivedDim.__init__cCsvddlm}ddlm}|jj| ur,| S|||jj}|j}|dksnJd|jd|jd|jdt|S)NrIntegerr;zExpected derived min value of z9 to be >= 0. Please specify an appropriate min value for (currently ).) rUrhr>r<rfr+rWr r6)r3rhr<Z _min_symintrfr'r'r(r+s    z_DerivedDim.minc Csddlm}ddlm}|jj|ur(|S|||jj}|j}|tjdks|Jd|j dtjdd|j d|jd t |S) Nrrgr;rzExpected derived max value of z to be <= z.. Please specify an appropriate max value for rirj) rUrhr>r<rfr,rWsysmaxsizer r6)r3rhr<Z _max_symintrfr'r'r(r,&s   z_DerivedDim.maxcstjfddS)Ncs|Sr.)rWrDrWr3r'r(rHArIz%_DerivedDim._derive..)rYrXrfrZr'rmr(rL:s  z_DerivedDim._derivecCs|jSr.)r r`r'r'r(raDsz_DerivedDim.__repr__N) r r!r"r#rVrrrBrer+r,rLrar'r'r'r(rYs   rYr9.)namesr+r,r0cstfdd|DS)zh Util to create multiple :func:`Dim` types. Returns: A tuple of :func:`Dim` types. c3s|]}t|dVqdS)r9N)r).0r:r,r+r'r( QrIzdims..)tuple)r+r,rnr'rpr(rHs rc@s"eZdZUdZeed<eed<dS)_ConstraintTargetz2 This represents input tensor dimensions. t_iddimN)r r!r"r#r6r5r'r'r'r(rsTs rsc@s`eZdZUdZeed<ded<dddZd d Zd d Zd dZ ddZ ddZ e ddZ dS) _Constraintz This represents a Dim describing a constraint target. `name` is the name of the Dim. `constraint_range` contains the min/max bounds of the Dim. r:rconstraint_rangerNcCs`ddlm}ddlm}ddlm}|dur0|}||jj|||d@dd}t|j |j |j |S)Nrrr; ValueRangeslowerupperFvrZ warn_only) %torch.fx.experimental.symbolic_shapesrr>r<torch.utils._sympy.value_rangesrzrwrrvrtrur:)r3r|r}rr<rzrwr'r'r(_clone_with_rangejs   z_Constraint._clone_with_rangecCs |j|dS)Nr|rr3r|r'r'r(__ge__~sz_Constraint.__ge__cCs|j|ddS)Nrrrrr'r'r(__gt__sz_Constraint.__gt__cCs |j|dS)Nr}rr3r}r'r'r(__le__sz_Constraint.__le__cCs|j|ddS)Nrrrrr'r'r(__lt__sz_Constraint.__lt__cCs tddS)NzCannot determine truth value of _Constraint. If you are trying to combine _Constraint's with logical connectives, you can specify them separately instead.)r2r`r'r'r(__bool__sz_Constraint.__bool__cCs|j|j|jjj|jjjdSN)rtrur+r,rtrurwrr|r}r`r'r'r(serializable_specs z_Constraint.serializable_spec)rN)r r!r"r#rVr5rrrrrrrerr'r'r'r(rv^s   rvc@s*eZdZUdZeed<ded<eed<dS) _PhantomRoota This represents the root of a derived Dim where the root does not directly specify the shape of any input dimension, but the derived Dim does. e.g., the input shapes 2*dim and dim + 1 are related via a "phantom" dim. The fields `name`, `constraint_range`, and `val` carried by a phantom root help create a symbol for it. Any derived dims with this phantom root are backed by expressions over this symbol. r:rrwvalN)r r!r"r#rVr5r6r'r'r'r(rs  rc@sFeZdZUdZeed<ded<eeefed<e ed<e ddZ d S) _DerivedConstraintaC This represents a derived Dim, whose root is either a regular constraint target (which directly specifies the shape of some input dimension) or a phantom root (which does so indirectly). It can be thought of as a subclass of `_Constraint`, except that it does not support <, <=, >, >= operations. r:rrwrfrWcCs|j|j|jjj|jjjdSrrr`r'r'r(rs z$_DerivedConstraint.serializable_specN) r r!r"r#rVr5r rsrrrerr'r'r'r(rs  rc@seZdZdZeddZdS)_RelaxedConstrainta This represents a dim marked with Dim.AUTO/DYNAMIC (i.e. mark_dynamic() or maybe_mark_dynamic()), which leaves relations & min/max ranges for inference, instead of requiring explicit specification. The intention is for constraint violations to not be raised if produce_guards() finds equalities or relations between a _RelaxedConstraint and another type of _Constraint. cCs|j|jdS)Nrtrurr`r'r'r(rsz$_RelaxedConstraint.serializable_specN)r r!r"r#rerr'r'r'r(rsrc@s<eZdZUdZeed<ejdddZe e e efed<dS) _IntWrapperz Dummy wrapper class to wrap around integer inputs so that when we parse the dynamic_shapes structure, we can mark if any of the integers were marked as dynamic. rFN)initdefaultdynamism) r r!r"r#r6r5 dataclassesfieldrrr r)r'r'r'r(rs rrr)rr)rrr) constraint get_sources shape_envrn source_pairsderived_equalitiesphantom_symbolsrelaxed_sourcescsJ||j|j}|sdS|^} |fdd| Dt|tr|j|vr|||j\} } || | } |fdd| Dn|j|jf||j<nt|tr0t|jts||jj|jjd} nX|jj|vr||jj} n>|j |jj t j j |jjt jjjjj|jjd} | ||jj<|j} || | fnt|trF|dS)z Updates `source_pairs`, `derived_equalities`, and `phantom_symbols` (which become fields of `EqualityConstraint`) based on a given input `constraint`. Nc3s|]}|fVqdSr.r'roZ other_sourcesourcer'r(rq rIz&_process_equalities..c3s|]}|fVqdSr.r'rrr'r(rqsr)rrZ dynamic_dimZconstraint_dim)rtruextend isinstancervr:rrfrZ create_symbolrtorch_dynamorZConstantSourceZfxZ experimentalZsymbolic_shapesZ DimDynamicr&rwrWappendradd)rrrrnrrrrsourcesZ other_sourcesZ shared_t_idZ shared_dimrfrWr'rr(_process_equalitiess:          r tree_name)functreedynamic_shapesrr0c sddfddzt|g|RdiWSty}zd|jdvr|s\JdshJd d d fd d fddt|d\}}|D]}t|\}}||gqWYd}~n d}~00dS)a" Customized tree_map for mapping pytrees to dynamic_shapes. For built-in types (e.g., standard collections) this behaves exactly like tree_map. OTOH for a user-defined class C registered with pytree, we cannot assume that a C containing tensors can be mapped to a C containing dynamic shapes (i.e., C may not be a polymorphic container). In that case we use the flattened form of C instead. Thus a C(**tensors) that flattens to (**tensors) will map to (**dynamic_shapes). Args: func: function to apply to each (int, float, str, bool, None, torch.Tensor) tree: input pytree dynamic_shapes: zero or more (typically one) dynamic_shapes to match Returns: output pytree mapping func to each (int, float, str, bool, None, torch.Tensor) cSs t|tvSr.)r r )tr'r'r(is_leafJsz$_tree_map_with_path..is_leafcsLt|}|tvr6tt||dg|RdiS||g|RSdS)Nrr)r rrZ flatten_fn)pathrrtyp)frrr'r(rQsz_tree_map_with_path..frZmismatchrz2Cannot be a mismatch if there is no dynamic_shapesz7Must provide a tree_name when there might be a mismatchcSsF|turt||S|ttfvr4|dus,Jt|Std|dS)NzDid not expect type )dictrlistrrrAssertionError)type_contextir'r'r(_keyis    z!_tree_map_with_path.._keycs0ddlm}m}||jdd|dddS)Nr UserError UserErrorTypez,Detected mismatch between the structure of `z` and `dynamic_shapes`: dynamic_shapes_validationZ case_name)torch._dynamo.excrr INVALID_INPUT)msgrrrr'r(raise_mismatch_errorrs z1_tree_map_with_path..raise_mismatch_errorc svt|}t|trdSt|trBd|d|jd|d|j|jkrvd|d|jd|d|j t|jt|jkrd|dt|jd|dt|jd |jtur.t|jt|jkrd|d|jd|d|j tt |j|jfd d |jD}n|j}t t |j|D],\}\}}||||j|j|gqDdS) N`z` is a z, but `dynamic_shapesz` is notz` has z elements, but `dynamic_shapesz elementsz ` has keys csg|] }|qSr'r')rokZ_remapr'r( rIz9_tree_map_with_path.._compare..) r rr r*lenZchildren_specsrsortedrzip enumerate)rrr rendered_pathZdynamic_shapes_children_specsrZtree_Zdynamic_shapes_)_comparerrrrr(r{s^      z%_tree_map_with_path.._comparerN)r ValueErrorargsr) rrrre_Z tree_specZ other_treeZother_tree_specr')rrrrrrrr(_tree_map_with_path1s    +rFr/cCs`t|tr|}|s\t|tjjr0t|jnt|}|durF|ni}|j |i|j S|Sr.) rrmodulernnModuleinspect signatureZforwardbind arguments)rrkwargsZ_is_torch_jit_tracerr'r'r( _combine_argss  rc@s:eZdZdZddZddZddZdd Zd d d Zd S)ShapesCollectiona* Builder for dynamic_shapes. Used to assign dynamic shape specifications to tensors that appear in inputs. This is useful particularly when :func:`args` is a nested input structure, and it's easier to index the input tensors, than to replicate the structure of :func:`args` in the :func:`dynamic_shapes` specification. Example:: args = {"x": tensor_x, "others": [tensor_y, tensor_z]} dim = torch.export.Dim(...) dynamic_shapes = torch.export.ShapesCollection() dynamic_shapes[tensor_x] = (dim, dim + 1, 8) dynamic_shapes[tensor_y] = {0: dim * 2} # This is equivalent to the following (now auto-generated): # dynamic_shapes = {"x": (dim, dim + 1, 8), "others": [{0: dim * 2}, None]} torch.export(..., args, dynamic_shapes=dynamic_shapes) To specify dynamism for integers, we need to first wrap the integers using _IntWrapper so that we have a "unique identification tag" for each integer. Example:: args = {"x": tensor_x, "others": [int_x, int_y]} # Wrap all ints with _IntWrapper mapped_args = pytree.tree_map_only(int, lambda a: _IntWrapper(a), args) dynamic_shapes = torch.export.ShapesCollection() dynamic_shapes[tensor_x] = (dim, dim + 1, 8) dynamic_shapes[mapped_args["others"][0]] = Dim.DYNAMIC # This is equivalent to the following (now auto-generated): # dynamic_shapes = {"x": (dim, dim + 1, 8), "others": [Dim.DYNAMIC, None]} torch.export(..., args, dynamic_shapes=dynamic_shapes) cCs i|_dSr.)_shapesr`r'r'r(rBszShapesCollection.__init__cCsnt|tjtfs"Jdt|t|}||jvr\|j|}||ksjJd|d|n||jt|<dS)Nz:Cannot assign shape to non-tensor or non-_IntWrapper type z0Shapes assigned to input do not match: expected z, got )rrTensorrr*idr)r3rshapertZ_shaper'r'r( __setitem__s    zShapesCollection.__setitem__cCs&t|}||jvri|j|<|j|Sr.)rr)r3rrtr'r'r( __getitem__s  zShapesCollection.__getitem__cCs t|jSr.)rrr`r'r'r(__len__szShapesCollection.__len__NcsNtfdd}t|||}t||}tfddjDrJtd|S)zu Generates the :func:`dynamic_shapes` pytree structure according to :func:`args` and :func:`kwargs`. cs.t|}|jvr&|j|SdSdSr.)rrr)rrrtr3t_idsr'r( find_shape s    z3ShapesCollection.dynamic_shapes..find_shapec3s|]}|vVqdSr.r')rort)rr'r(rqrIz2ShapesCollection.dynamic_shapes..zSome tensors that were assigned shapes were not found in args. Maybe such tensors were copied when passing them as args? Maybe such tensors are contained in classes that were not registered with pytree?)setrranyrr)r3mrrr combined_argsrr'rr(rs  zShapesCollection.dynamic_shapes)N) r r!r"r#rBrrrrr'r'r'r(rs (rc@s4eZdZdZddZd ddZd ddZd d ZdS) raQ Infers dynamic_shapes based on additional inputs. This is useful particularly for deployment engineers who, on the one hand, may have access to ample testing or profiling data that can provide a fair sense of representative inputs for a model, but on the other hand, may not know enough about the model to guess which input shapes should be dynamic. Input shapes that are different than the original are considered dynamic; conversely, those that are the same as the original are considered static. Moreover, we verify that the additional inputs are valid for the exported program. This guarantees that tracing with them instead of the original would have generated the same graph. Example:: args0, kwargs0 = ... # example inputs for export # other representative inputs that the exported program will run on dynamic_shapes = torch.export.AdditionalInputs() dynamic_shapes.add(args1, kwargs1) ... dynamic_shapes.add(argsN, kwargsN) torch.export(..., args0, kwargs0, dynamic_shapes=dynamic_shapes) cCs g|_dSr.) _examplesr`r'r'r(rB7szAdditionalInputs.__init__NcCsTt|tusJd|d|dus@t|tus@Jd|d|j||fdS)zC Additional input :func:`args` and :func:`kwargs`. zRepresentative args z must be a tupleNzRepresentative kwargs z must be None or a dict)r*rrrrr)r3rrr'r'r(r:s  zAdditionalInputs.addcsFfdd||fg|jD^}}dd}t||g|RdddiS)z Infers a :func:`dynamic_shapes` pytree structure by merging shapes of the original input :func:`args` and :func:`kwargs` and of each additional input args and kwargs. cs&g|]\}}tddt||qS)cSst|tjrt|jS|Sr.)rrrrrr)rrr'r'r(rHNrIz..)rr)rorrrr'r(rLs  z3AdditionalInputs.dynamic_shapes..cstfdd|Ds,tdf|dttrbttsbtfdd|DrZdStjSn0tfdd|Dstdf|ddSdS)Nc3s|]}tt|kVqdSr.)r*)rorGvr'r(rqUrIzJAdditionalInputs.dynamic_shapes.._mark_dynamism..z^The following inputs were found to have differing types, so they cannot be marked as dynamic: .c3s|]}|kVqdSr.r'roZother_vrr'r(rq\rIc3s|]}|kVqdSr.r'rrr'r(rqarIz`The following inputs were found to have differing values, but they cannot be marked as dynamic: )allrrr6r7rr&)rZother_vsr'rr(_mark_dynamismTs"z7AdditionalInputs.dynamic_shapes.._mark_dynamismrcSs t|tuSr.)r*r6)rr'r'r(rHlrIz1AdditionalInputs.dynamic_shapes..)rr)r3rrrrZother_dynamic_shapesrr'rr(rEs  zAdditionalInputs.dynamic_shapescCs2|}|jD]\}}tjj|||p(iqdS)zW Verifies that an exported program is valid for each additional input. N)rrrexportZ_unliftZ!_check_input_constraints_pre_hook)r3epZepmrrr'r'r(verifyos  zAdditionalInputs.verify)N)N)r r!r"r#rBrrrr'r'r'r(rs  *rcCsd}t|dS)NzTUsing None as a dynamic shape dimension is deprecated. Please use Dim.STATIC instead)logwarning)rr'r'r(%_warn_on_None_dynamic_shape_dimension{sr)rrcs6ddlmm|dus$t|dkr(dSt|ttfrFt||}ifddfddt|t ttfs~Jt|t rt| }t| }t |t |krd|d |d }t|d kr|d|vrt||dt r|d 7}n|d 7}j |ddfdd}t |||dddS)z Checks the dynamic_shapes specification for correctness, using combined args + kwargs as reference for inputs structure. rrNc s|jvrr|j\}}|j|ks,|j|krt|j||}t|j|j|j}jd|d|d|dn|j|jf|j<dS)NzFound different definitions z and z! for the same symbolic dimension !)r r+r,rr_r)rur[r\Zthis_Zthat_)rrboundsr'r(check_same_boundss z0_check_dynamic_shapes..check_same_boundsc st|tr|D]h\}}t|tr.|q|dur>tqt|ttfsjd|d|dt|d|d ddqnt|t t frZt |t |j krjd|dt|d |j d t |j d t |d ddt |D]l\}}t|tr|q|durtqt|ttfsjd |d|dt|d |d ddqn,|durjd|dt|ddddS)Nz%Unexpected dimension mapped to index z in input tensor shape  specified at `dynamic_shapeszP` (expected None, an int, a Dim, Dim.AUTO, Dim.STATIC, or Dim.DYNAMIC, but got z instead)rrzExpected dynamic shape spec z5` to have the same length as the actual tensor shape z (expected z , but got zUnexpected dimension #zO` (expected None, an int, a Dim, Dim.AUTO, Dim.STATIC, or Dim.DYNAMIC, but got zUnexpected input tensor shape z` (expected either a list/tuple of dimensions, or a dict mapping indices to dimensions, where each dimension is an int, a Dim, Dim.AUTO, Dim.STATIC, or Dim.DYNAMIC))rritemsrrr6r)rr rrrrrrrtensorrrru)rrrr'r( check_symbolssd         z,_check_dynamic_shapes..check_symbolszWWhen `dynamic_shapes` is specified as a dict, its top-level keys must be the arg names z of `inputs`, but here they are z. rzSince here `inputs` is a list/tuple enclosing a single dict, maybe you just forgot to enclose `dynamic_shapes` in a list/tuple?zwAlternatively, you could also ignore arg names entirely and specify `dynamic_shapes` as a list/tuple matching `inputs`.rrc st|tjr|||nvt|trRt|tr6td|dust|ttfsJn>|durt|}j d|d|dt |d|d dddS) NzdUnable to specify input integers as dynamic through named Dims. Please use Dim.AUTO/DYNAMIC instead.zCannot associate shape rz` to non-tensor type z at `inputsz` (expected None)rr) rrrr_Dimrr6r)r rr*)rr dynamic_shaper)rrrr'r( check_shapes&   z*_check_dynamic_shapes..check_shapeinputsr)rrrrrrrrr*valuesrkeysrrr)rrZgot_keysZexpected_arg_namesrrr')rrrrrr(_check_dynamic_shapessH0     r )rrr0csddlmm|dus$t|dkr(gSt|ttfrFt||}t tiggfddddfdd fd d fd d }t |||d dD]"}|j j }|vr|d|_ qD]} |q܈S)zT Reads the dynamic_shapes specification and produces a list of constraints. rrNc s|ddlddlm}ddlmddlm}fdd}ttr j }|j vr~ |j d}t |j |j }nF|j vrt|j |||j|jddd |d }||j <n |j }ttj ||jjddd |j} t|trx| nnttr@ttj ||jjddd } n8ttsPJttj ||jjddd } | S) Nrrx) try_solveryc sxjjjdd}|}|j|}|durHt|dSjddjd|d|d dS) NT)integerrzExpected shape[z] = z# of input Tensor to be of the form z, where z is an integer)rrfr rWEqrr6ZCONSTRAINT_VIOLATION)symbolexprZsolution)rrrurrUrr r'r( root_value$s  zB_process_dynamic_shapes..to_constraint..root_valuer{Fr~)r:rwr)rUrrZtorch.utils._sympy.solver rrzrrYrfr rsrtrurr+r,rrrWrrbrvrdr) rurrrrzrZdim_rootZroot_constraintrfr)rr%derived_constraints_with_phantom_root phantom_rootssymbols)rurrUrr r( to_constraintst             z._process_dynamic_shapes..to_constraintr/csdd}t|ttfrJt|tr,||||}|||}|j|nt|tr|jtjkrpt j ||n6|jtj krt j ||n|jtjkrt j ||tt||n|durt j ||dS)NcSst|Sr.)rb)rrrdr'r'r(_create_static_dimsszN_process_dynamic_shapes.._parse_tensor_dim.._create_static_dim)rr6rr rr)r*rr$rrZmaybe_mark_dynamicr%Z mark_staticr&Z mark_dynamicrr)ridxrurr) constraintsrrr'r(_parse_tensor_dimrs        z2_process_dynamic_shapes.._parse_tensor_dimcst|_t|_t|_t|_t|_t|trR|D]\}}|||q:nTt|t t frt |D]\}}|||qhn&|durt | D]}||dqdSr.)rZ_dynamo_weak_dynamic_indicesZ_dynamo_dynamic_indicesZ_dynamo_dynamic_rangeZ_dynamo_static_indicesZ_dynamo_unbacked_indicesrrrrrrrrangerur)rr'r(update_symbolss z/_process_dynamic_shapes..update_symbolscs.t|tjr|||nt|tr*||_dSr.)rrrrr)rrr)rr'r( assoc_shapes  z,_process_dynamic_shapes..assoc_shaperr)rrrrrrrrr*r rrrfr:r)rrrZ$derived_constraint_with_phantom_rootZphantom_root_nameZ dynamic_dimsr') rrrrrrrrrr(_process_dynamic_shapess*U    r)rcCsvi}t|ddddD]X}|dur&qt|tr4qqt|trb|||j<t|trp|j||jj<qt|tsJq|S)NcSs t|tSr.)rrrDr'r'r(rHrIz'_get_dim_name_mapping..rr)rrr6rr rYrfr))r name_to_dimrur'r'r(_get_dim_name_mappings"    r)rrr0c s"ddl}ddlddlm}m}ddlm}z|dd}Wn2t yt}z||j d|WYd}~n d}~00i|dD]}|}| d |} r| d} d \} } | d |} rt | d} | d |}rt | d} t| | | d | <q|d\} }|}t|jr8t || <q|| <qt|t}D]z\}}t|t ttjfszJt|jr||sJ||<|ttt|jt|trX||jjqXD]"\}}|vs||vsJqifdd}t|||S)a When exporting with :func:`dynamic_shapes`, export may fail with a ConstraintViolation error if the specification doesn't match the constraints inferred from tracing the model. The error message may provide suggested fixes - changes that can be made to :func:`dynamic_shapes` to export successfully. Example ConstraintViolation error message:: Suggested fixes: dim = Dim('dim', min=3, max=6) # this just refines the dim's range dim = 4 # this specializes to a constant dy = dx + 1 # dy was specified as an independent dim, but is actually tied to dx with this relation This is a helper function that takes the ConstraintViolation error message and the original :func:`dynamic_shapes` spec, and returns a new :func:`dynamic_shapes` spec that incorporates the suggested fixes. Example usage:: try: ep = export(mod, args, dynamic_shapes=dynamic_shapes) except torch._dynamo.exc.UserError as exc: new_shapes = refine_dynamic_shapes_from_suggested_fixes( exc.msg, dynamic_shapes ) ep = export(mod, args, dynamic_shapes=new_shapes) rNr)_is_supported_equivalencezSuggested fixes:rz`Suggested fixes not found in error message given to refine_dynamic_shapes_from_suggested_fixes() z(.*) = Dim\('(.*)'.*\))NNz!.* = Dim\('.*', min\=([0-9]+).*\)z.* = Dim\('.*'.*max\=([0-9]+)\)r9z = c s6|dust|tr|S|jvr|j}t|jrt|vrNt|Stt|j}|jvrr|j}n|jvsJ|j}j j ||\}}|}|dkrt||}|dkr|t|}|t|<|Sn|SnPt|t r2|j jvr2|jvr|jS||j j}||j<|S|S)Nrr)rr6r ExprrVnextiter free_symbolsr:ZpolysZ polytoolsdivrYrfrW) rrudummyfixrrfmodulus remainderZ_dimZderived_dim_cacherZ shape_fixesrUr'r( apply_fixes's8              z?refine_dynamic_shapes_from_suggested_fixes..apply_fixes)rerUrrrrr splitstrip Exceptionrmatchgroupr6rrTrNumberrrrrYr"rrVr#r$r%rfr r)rrr-rrr Zshape_fixes_msgexcr(r1r:r@rAZ match_minZ match_maxrrootsrcr,r'r+r(rsX      $r)F)Grrloggingrk collectionsrenumrrtypingrrrrr rZtorch.utils._pytreer r r r rrrrrrZexported_programrrUrZ torch._guardsrrrr__all__ getLoggerr rrZ dataclassr)rrrbrYrVr6rrrrsrvrrrrrrrrrrrrrrr rrrr'r'r'r(s 0     L  G   ?  |__    4