eThNSSKrSSKJr SSKJrJr SSKJr SSKJ r SSK J r J r SSK JrJr SSKJr SS KJrJrJr \ "S 5r/S Qr"S S \5rS\S\S\4SjrS\ 4SjrSrS\S\S\\\\\\\44Sjr S1S\S\S\S\ \S\4 Sjjr!S\S\"S\S\#4Sjr$S\S\"S\4S jr%S2S\S\S\S\"S\ \S\#S\4S!jjr&S"\S\4S#jr'S"\S\4S$jr(S%\4S&jr)\*S':XGan\"5r+\+RY5r-\"\-R\5R_5\-l.\0"S(5 \&"\-Rb\-Rd\-R\\-Rf\-Rh\-Rj\-R5 \-RP(aV\"\5 \-RbS):Xa\0"S*5 \0"S+5 \'"\-R\5\-l6\("\-Rl5\-l7\-Rp(a\\0"S,5 \)"\-R\5 \9"\-S-5(a\)"\-Rl5 \9"\-S.5(a\)"\-Rn5 gggg!\:ar;\0"S/\;35 \<"S05 Sr;C;gSr;C;ff=f)3N)ArgumentParser)listdirmakedirs)Path)Optional)Versionparse)Pipelinepipeline) BatchEncoding) ModelOutputis_tf_availableis_torch_availablez1.4.0) feature-extractionnerzsentiment-analysisz fill-maskzquestion-answeringztext-generationtranslation_en_to_frtranslation_en_to_detranslation_en_to_roc,^\rSrSrSrU4SjrSrU=r$)OnnxConverterArgumentParser.zS Wraps all the script arguments supported to export transformers models to ONNX IR c>[TU]S5 URS[[SS9 URS[SSS9 URS [S S 9 URS [S S/SS9 URS[ SSS9 URSSSS9 URSSSS9 URSSSS9 URS5 g)NzONNX Converterz --pipeliner)typechoicesdefaultz--modelTz4Model's id or path (ex: google-bert/bert-base-cased))rrequiredhelpz --tokenizerz8Tokenizer's id or path (ex: google-bert/bert-base-cased))rrz --frameworkpttfzFramework for loading the model)rrrz--opset zONNX opset to use)rrrz--check-loading store_truez$Check ONNX is able to load the model)actionrz--use-external-formatz!Allow exporting model >= than 2Gbz --quantizez/Quantize the neural network to be run with int8output)super__init__ add_argumentstrSUPPORTED_PIPELINESint)self __class__s Z/var/www/auris/envauris/lib/python3.13/site-packages/transformers/convert_graph_to_onnx.pyr%$OnnxConverterArgumentParser.__init__3s )*  '(   G  -c8rs  4L2  )#r@ST  7   #4   B  (#)__name__ __module__ __qualname____firstlineno____doc__r%__static_attributes__ __classcell__)r+s@r,rr.s&$&$r.rfilename identifierreturncURRURU-5RUR5$)a- Append a string-identifier at the end (before the extension, if any) to the provided filepath Args: filename: pathlib.Path The actual path object we would like to add an identifier suffix identifier: The suffix to add Returns: String with concatenated identifier at the end of the filename )parentjoinpathstem with_suffixsuffix)r7r8s r,generate_identified_filenamer@\s1 ?? # #HMMJ$> ? K KHOO \\r.minimum_versioncSSKn[UR5nU[:a[ SURSUS35eg![a [ S5ef=f)z Check onnxruntime is installed and if the installed version match is recent enough Raises: ImportError: If onnxruntime is not installed or too old version is found rNz*We found an older version of onnxruntime (z&) but we require onnxruntime to be >= zp to enable all the conversions options. Please update onnxruntime by running `pip install --upgrade onnxruntime`zonnxruntime doesn't seem to be currently installed. Please install the onnxruntime by running `pip install onnxruntime` and relaunch the conversion.) onnxruntimer __version__ORT_QUANTIZE_MINIMUM_VERSION ImportError)rArC ort_versions r,check_onnxruntime_requirementsrHis K334  5 5<[=T=T[U[[45(aUVs/sH nT "XX#5PM sn$[UR5VVs/sHupVUS:XdM UPM snnSS0nU(aA[ UR5S:XaSUS'Ow[ S[ UR5S35e[UR5VV s/sHupX:XdM UPM n nn UR[RU S55 [SU(aS OS S US U35 U$s snfs snnfs sn nf) NrJrbatchsequencezUnable to infer tensor axes ()zFound inputr# z with shape: ) isinstancerPlist enumerateshapelen ValueErrorupdatedictfromkeysrK) r\tensorr]r^taxisnumelaxesdimriseq_axesbuild_shape_dicts r,rv&infer_shapes..build_shape_dicts fudm , ,JPQ&Q$Th@&Q Q.7v||-DS-DkdQR T-DSTUVX_`Dv||$)(DG$'DSEVDWWX%YZZ2;FLL2I^2IJCUM]C2I^ DMM(J?@ (w94& dVTU RT_sD* D/D/ D5D5zThis is a sample output)return_tensorsrToutput_Fr/)r'boolr) tokenizer input_idsrirQrfr to_tuplergrPkeysitemsextendrOrangerjziprm)rYrZrRr^outputs input_varskvinput_dynamic_axes outputs_flatr#i output_namesoutput_dynamic_axes dynamic_axesrvs @r, infer_shapesrs sdS&]]4Y] OF$$R(G%.$%6cii!&!CIIf 2Gb /!\rN)rDzUsing framework TensorFlow: z , tf2onnx: r)r\)r output_pathzCannot import z6 required to convert TF model to ONNX. Please install z first. )rrrK tensorflowtf2onnxrDversionVERSIONrrQpredictdatar TensorSpec from_tensorconvert from_kerasrrFr\)rYrr#rrt2ovrSrrrRkeyroinput_signature model_proto_es r,convert_tensorflowr&s*   fgg RS / ,RZZ-?-?,@ D6RS;GsD:Q7 < &++&Z`ZfZfZhiZh;32==44V4FZhi 33 IIAR4  Qj  QVVH$Z[\[a[aZbbjkljm n   s*A:D!%C?8D?D D<(D77D<c [R"S[5 [SU35 [ X`X40UD6nUR R 5(d<[SUR 35 [UR R55 OW[[UR R555S:a'[SUR R5S35eUS:Xa [XX%5 g[XU5 g) a Convert the pipeline object to the ONNX Intermediate Representation (IR) format Args: framework: The framework the pipeline is backed by ("pt" or "tf") model: The name of the model to load for the pipeline output: The path where the ONNX graph will be stored opset: The actual version of the ONNX operator set to use tokenizer: The name of the model to load for the pipeline, default to the model's name if not provided use_external_format: Split the model definition from its parameters to allow model bigger than 2GB (PyTorch only) pipeline_name: The kind of pipeline to instantiate (ner, question-answering, etc.) model_kwargs: Keyword arguments to be forwarded to the model constructor Returns: zoThe `transformers.convert_graph_to_onnx` package is deprecated and will be removed in version 5 of TransformerszONNX opset version set to: zCreating folder rzFolder z" is not empty, aborting conversionrN)warningswarn FutureWarningrKrr;existsrrrjrrrr) rZrQr#rr|rrrrYs r,rrNs6 MM   'w /0 } Z\ ZC ==   ! !  01'')* WV]]++- . /! 3'&--"8"8":!;;]^__DF@3v.r.onnx_model_pathcSSKJnJn [US5nU"5nUR 5UlU"UR 5U5n[ SUS35 [ S5 U$)a& Load the model at the specified path and let onnxruntime look at transformations on the graph to enable all the optimizations possible Args: onnx_model_path: filepath where the model binary description is stored Returns: Path where the optimized model binary description has been saved rInferenceSessionSessionOptionsz -optimizedz$Optimized model has been written at : ✔zY/!\ Optimized model contains hardware specific operators which might not be portable. /!\)rCrrr@roptimized_model_filepathrK)rrropt_model_path sess_optionrs r,optimizersf=2/<PN "K+9+B+B+DK(113[AA 00@@V WX gh r.cSSKnSSKnSSKJn SSKJn SSKJn SSKJ n URUR55n[UR5[S5:a [S5 U"5nURU5 [UR5[S 5:a$U"US S UR S S S SSS[#U5S 9 n O#U"US S UR S S S SSS[#U5S 9 n U R%5 ['US5n [SU S35 UR)U R*R*U R55 U $)z Quantize the weights of the model from float32 to in8 to allow very efficient inference on modern CPU Args: onnx_model_path: Path to location the exported ONNX model is stored Returns: The Path generated for the quantized rN) ModelProto)QuantizationMode) ONNXQuantizer)IntegerOpsRegistryz1.5.0zpModels larger than 2GB will fail to quantize due to protobuf constraint. Please upgrade to onnxruntime >= 1.5.0.z1.13.1FT) rQ per_channel reduce_rangemodestatic weight_qType input_qType tensors_rangenodes_to_quantizenodes_to_excludeop_types_to_quantize) rQrrrrractivation_qTyperrrrz -quantizedz$Quantized model has been written at r)onnxrC onnx.onnx_pbronnxruntime.quantizationr'onnxruntime.quantization.onnx_quantizerr!onnxruntime.quantization.registryrloadrr rDrKCopyFrom IntegerOpsrgquantize_modelr@ save_modelrQ) rrrCrrrr onnx_model copy_model quantizerquantized_model_paths r,quantizersO'9ED?3356J T  w/  6 J #  [ $ $%h7!!,,"!!%&8!9  "!,,""!!%&8!9  8V 01E0FF\ ]^OOIOO))+?+H+H+JK r.pathcSSKJnJn SSKJn [ SUS35 U"5nU"UR 5US/S9n[ SUS 35 g!Uan[ S US 35 SnAgSnAff=f) Nrr)RuntimeExceptionz"Checking ONNX model loading from: z ...CPUExecutionProvider) providerszModel u correctly loaded: ✔zError while loading the model u: ✘)rCrr+onnxruntime.capi.onnxruntime_pybind11_staterrKr)rrrr onnx_optionsrres r,verifyrsz<L .tfD 9:I%' T]]_lG]F^ _ tfCDE I .rd2FGHHIs-A A,A''A,__main__z' ====== Converting model to ONNX ======raV Using TensorFlow might not provide the same optimization level compared to PyTorch. For TensorFlow users you can try optimizing the model directly through onnxruntime_tools. For more information, please refer to the onnxruntime documentation: https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/python/tools/transformers z$ ====== Optimizing ONNX model ======z+ ====== Check exported ONNX model(s) ======optimized_outputquantized_outputz"Error while converting the model: rJ)N)NFr)=rargparserosrrpathlibrtypingrpackaging.versionrr transformers.pipelinesr r transformers.tokenization_utilsr transformers.utilsr rrrEr(rr'r@rHrXrPrgrmrrr)r{rrrrrrr0parser parse_argsargsr#absoluterKrZrQrr|rrr check_loadinghasattrrrexitr/r.r,rs+# ,59OO %W~ +$.+$\ ]4 ]S ]T ] G :28=:h=:3=:5cDItUb9b3c=:BPTvv#&v/2v?G}v v>" " #" t" RV" J% H% S% $% Z $ %-//// // //  // } //  ////ddt2H dH tH V I I z ( *F    Dt{{#,,.DK/ 89 NN JJ KK JJ NN  $ $ MM  == *+G H~~%w 9 :%-T[[$9D !%-T-B-B$CD !    @ A 4;; t/00t,,-t/00t,,-1 Uh  21#67 Qs(D.II= I88I=