fThSSKJrJrJrJr SSKrSSKJr SSKJr SSK J r J r J r J r SSKJr SSKJr SS KJr SS KJr SS KJr SS KJrJrJrJr SS KJrJr SSK J!r!J"r" SSK#J$r$ SSK%J&r&J'r'J(r(J)r)J*r* SSK+J,r, \)"5(a SSK-J.r. SSK/J0r0 \*Rb"\25r3"SS\Rh5r5Sr6S\RnS\8S\Rn4Sjr9S=S\RhS\RnS\RnS\RnS \\RnS!\:S"\:4S#jjr;S>S$jr<"S%S&\Rh5r=\"S'5"S(S)\Rh55r>"S*S+\5r?\'"S,S-\"55r@"S.S/\Rh5rA\'"S0S1\@55rB"S2S3\\&5rC\'"S4S5\@\55rD\'"S6S79"S8S9\@55rE\'"S:S;\@55rF/S[TU]5 Xl[R"UR SUR -SS9Ul[R"UR UR SS9Ul[URUl g)NFbias) super__init__configrLinear hidden_sizeintermediate_size gate_up_proj down_projr hidden_act activation_fnselfr. __class__s ^/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/phi3/modeling_phi3.pyr-Phi3MLP.__init__:sn  IIf&8&8!f>V>V:V]bc6#;#;V=O=OV[\#F$5$56 hidden_statesreturncURU5nURSSS9up2X RU5-nURU5$)Nr)dim)r2chunkr5r3)r7r< up_statesgates r9forwardPhi3MLP.forwardBsH%%m4 #//!/4 2 24 88 ~~i((r;)r5r.r3r2) __name__ __module__ __qualname____firstlineno__r-torch FloatTensorrE__static_attributes__ __classcell__r8s@r9r&r&9s,7)U%6%6)5;L;L))r;r&cUSSURSS-24nUSURSS-S24n[R"U*U4SS9$)z*Rotates half the hidden dims of the input..Nr?r)r@)shaperKcat)xx1x2s r9 rotate_halfrVKsZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r;r<n_repr=cURup#pEUS:XaU$USS2SS2SSS2SS24RX#XU5nURX#U-XE5$)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) r!N)rQexpandreshape)r<rWbatchnum_key_value_headsslenhead_dims r9 repeat_kvr_Rs_ 2?1D1D.E z!!Qa"23::5W\dlmM  e(CT TTr;modulequerykeyvalueattention_maskscalingdropoutc @[X R5n[X0R5n [R"XR SS55U-n Ub"USS2SS2SS2SUR S24n X-n [ RRU S[RS9RUR5n [ RRXURS9n [R"X5n U R SS5R5n X4$)Nr)rr?)rAdtype)ptrainingr!)r_num_key_value_groupsrKmatmul transposerQr functionalsoftmaxfloat32torirfrk contiguous) r`rarbrcrdrerfkwargs key_states value_states attn_weights causal_mask attn_outputs r9eager_attention_forwardrz^s3 ; ;[TU]5 XlX l[ USUR UR -5UlUR UR-Ul URUlURS-Ul URUl SUl UR UR-SURUR---n[R"UR UR-UR SS9Ul[R"UR USS9Ulg)Nr^gTr)Fr*)r,r-r.rgetattrr0num_attention_headsr^r\rlreattention_dropout is_causalrr/o_projqkv_proj)r7r.rop_sizer8s r9r-Phi3Attention.__init__s  " F4F4F&JdJd4de $*$>$>&B\B\$\!#)#=#= }}d* !'!9!9,,t}}d>d##L '>dkk>^>^&_#$7      % $}}C$2H2HLL"4;;0@$G %  % ! \"));;;;FFH kk+. L((r;) rr.r^rrrlr\rrreN)NN)rGrHrIrJ__doc__r"rintr-rKTensorrr LongTensorrrrErMrNrOs@r9rrsG Kz Khsm K K(+/59 6)||6)#5<<#=>6)!. 6) ! 6) !!1!12 6)-.6) u||Xell3XeELL>Q5RR S6)6)r;rRMSNormc8^\rSrSrSU4SjjrSrSrSrU=r$) Phi3RMSNormc>[TU]5 [R"[R "U55UlX lg)z* Phi3RMSNorm is equivalent to T5LayerNorm N)r,r-r ParameterrKonesweightvariance_epsilon)r7r0epsr8s r9r-Phi3RMSNorm.__init__s/ ll5::k#:; #r;cURnUR[R5nUR S5R SSS9nU[R "X0R-5-nURURU5-$)Nr)r?T)keepdim) rirrrKrqpowmeanrsqrtrr)r7r< input_dtypevariances r9rEPhi3RMSNorm.forwardsw#)) %((7  $$Q',,R,>% H?T?T4T(UU {{]--k:::r;c^[URR5SUR3$)Nz, eps=)tuplerrQrr7s r9 extra_reprPhi3RMSNorm.extra_reprs* ))*+6$2G2G1HIIr;)rr)gư>) rGrHrIrJr-rErrMrNrOs@r9rrs$;JJr;rc^\rSrSrS\S\4U4SjjrSS\RS\ \RS\ \RS\ \ S \ \ S \ \ S \ \RS \ \ \R\R4S \\S\ \R \ \ \R \R 444SjjrSrU=r$)Phi3DecoderLayerr.rc>[TU]5 URUl[XS9Ul[ U5Ul[URURS9Ul [URURS9Ul Xl [R"UR5Ul[R"UR5Ulg)N)r.rr)r,r-r0r self_attnr&mlpr rms_norm_epsinput_layernormpost_attention_layernormr.rDropout resid_pdropresid_attn_dropoutresid_mlp_dropoutr7r.rr8s r9r-Phi3DecoderLayer.__init__s !--&fJ6?*6+=+=6CVCVW(3F4F4FFL_L_(`% "$**V-?-?"@!#F,>,>!?r;r<rdrrr use_cacherrrtr=c Un URU5nUR"SUUUUUUUUS.U D6upXRU5-nUn URU5nUR U5nXR U5-nU4n U(aX4- n U $)a5 Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. position_ids (`torch.LongTensor` of shape `({0})`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) past_key_value (`Cache`, *optional*): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence kwargs (`dict`, *optional*): Arbitrary kwargs to be ignored, used for FSDP and other methods that injects code into the model )r<rdrrrrrr)rrrrrr) r7r<rdrrrrrrrtresidualself_attn_weightsoutputss r9rEPhi3DecoderLayer.forwardsD!,,]; ,0>> , ')%)/) 3 ,  , ( !#:#:=#II  55mD /  #9#9-#HH  "  + +Gr;)r.r0rrrrrr)NNNFFNN)rGrHrIrJr"rr-rKrrrr boolrrrrLrErMrNrOs@r9rrs @z @c @2637*.,1$)59KO=||=!.=u//0 = ! = $D> =D>=!!1!12=&eELL%,,,F&GH=-.= u  (51B1BEDUDU1U+V"WW X==r;rcR\rSrSr\rSrSrS/rS/r Sr Sr Sr Sr SrSrSrSrSrSrg ) Phi3PreTrainedModeliDmodelTrpast_key_valuesz0.0.5cURRn[U[R5(aWUR R RSUS9 URb%URR R5 gg[U[R5(adUR R RSUS9 URb2UR R URR5 gg[U[5(a&UR R RS5 gg)Nr)rstdg?)r.initializer_range isinstancerr/rdatanormal_r+zero_ Embedding padding_idxrfill_)r7r`rs r9 _init_weights!Phi3PreTrainedModel._init_weightsTskk++ fbii ( ( MM   & &CS & 9{{&   &&('  - - MM   & &CS & 9!!- ""6#5#56<<>.  , , MM   $ $S )-r;rN)rGrHrIrJr" config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_2_supports_sdpa_supports_flex_attn_supports_cache_class_supports_quantized_cache_supports_static_cache_supports_attention_backend_versionrrMrr;r9rrDsXL&*#+,#4"5!N  $!"&H *r;rcl^\rSrSrSS\4U4Sjjjr\R"5\S55r Sr U=r $)Phi3RotaryEmbeddingibr.c>[TU]5 [US5(aHURb;URR SURR S55UlOSUlUR UlUR UlXl [UR Ul URURU5uo0l URSUSS9 URUlg)N rope_scaling rope_typetypedefaultinv_freqF) persistent)r,r-hasattrrrrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenr.r rope_init_fnattention_scalingregister_bufferroriginal_inv_freq)r7r.devicerr8s r9r-Phi3RotaryEmbedding.__init__cs  6> * *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E'E!((--[`J`ahhmmfk ^^ U C&,,.1F1L1L1NNYYZ[]^_E))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs $BF  F.)rr.r rr rrr) rGrHrIrJr"r-rKno_gradrrErMrNrOs@r9rrbs6/z//" ]]_ < [TU]U5 URUlURUl[ R "URURUR5Ul[ R"[UR5Vs/sHn[X5PM sn5Ul [URURS9Ul[#US9UlSUlUR)5 gs snf)Nr)r.F)r,r- pad_token_idr vocab_sizerrr0 embed_tokens ModuleListrangenum_hidden_layersrlayersrrnormr rotary_embgradient_checkpointing post_initrs r9r-Phi3Model.__init__s  !.. ++LL):):F??ab b   --i8M 0*nO  !CRC^==?de "\\ ]5H5H5K"KTaThThN  )33A6L.. >L] & #oomJ#7BD0d![[)H4;;+H+HIM#!%55!) *)."3#-$7 $ M*!,M  =#3"55'J* -0  !1 1 &+/8Od+%   r;r# input_tensorc URRS:Xa]UbNUbKUSS2S4R5R5UR 5S:gnU(a [ S5eUbSU;aU$gURRS:Xa,[ U[R5(a [U5nU$UbUR5OSn[ U[5n[ U[5n URRS:XaQU(dJU (dCU(d<[R"UUUURRUR S9(agUR"n [R$"U 5R&n UR(S n U (dU(aUR+5n O5[ U[R5(aUR(SOX|-S -n UR-UU U U UUR(SURUS 9nURRS:Xa:Ub7UR.R0S ;aU(d[R2"X5nU$) Nflash_attention_2r?rzYou are attempting to perform batched generation with padding_side='right' this may lead to unexpected behaviour for Flash Attention version of Phi3. Make sure to call `tokenizer.padding_side = 'left'` before tokenizing the input. rflex_attentionr)r:past_key_values_lengthr is_trainingr!)sequence_length target_lengthrir batch_sizer.r)cudaxpunpu)r.rsumitemsizerArrKrr$rBr r r_ignore_causal_mask_sdparrkrifinfominrQget_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionrr_unmask_unattended)r7rdrKrrris_padding_rightrEusing_static_cacheusing_sliding_window_cacheri min_dtyperQrRrxs r9rDPhi3Model._update_causal_masks; ;; + +/B B)o.I#1!R%#8#<#<#>#C#C#EIZIZI\]^I_#_ #$a )c^.C%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`a'E%/AS%T" KK , , 6'+E%%>>*'7#{{99 MM ""KK&** &,,Q/ %);+??AM nell;;$$R(%7!; PP +')#))!,;;+Q   KK , , 6*%%**.DD% 1CCK[Kr;rQrRrirScUbUR5S:XaUnU$[R"U5Rn [R"X4XUR S9n[R "X$R S9URSS5:n UR5n [U SS5(avU Rbi[U[5(aX:aO[R "X$R S9URSS5U R- :*n U RU 5 X-nUSSSS2SS24RUSSS5nUbUR5nUR SU:a USS2SU24nUR Sn USS2SS2SS2SU 24USS2SSSS24R#UR 5-nUS :HnUSS2SS2SS2SU 24R%X5USS2SS2SS2SU 24'U$) aP Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. config (`Phi3Config`): The model's configuration class past_key_values (`Cache`): The cache class that is being used currently to generate N) fill_valuerirr>r?r!use_sliding_windowTr)rArKr[r\fullrrCrZget_text_configrrrr bitwise_or_rYclonerQrr masked_fill)rdrQrRrirrSr.rrxrcdiagonal_attend_mask text_configsliding_attend_mask mask_length padding_masks r9r^?Phi3Model._prepare_4d_causal_attention_mask_with_cache_positionRsB  %.*<*<*>!*C(K@= E*..I** 0Y\j\q\qK$)<< F[F[#\_m_u_uA`$ !002K{$8$??KD^D^Dj"/3EFF/Ji*/,,}MbMb*c&..r15 8R8RR+')445HI  /K%dD!Q&67>>z1bRTUK))//1 !''+m;%3A~ ~4E%FN,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r;)r'r.r+r,rr-r& NNNNNNNNN)F)rGrHrIrJr"r-r3r7rrrrKrrr rLrrrrrErrD staticmethodrrir^rMrNrOs@r9r#r#sz !"151537+/59$(,0/359\ E,,-\ !.\ u//0 \ "% \   1 12 \ D>\ $D>\ 'tn\ !!1!12\ $$89\  !\ \ H#( TellK78TllT T  T  TlB BBB{{ B  B  BBBBr;r#c\rSrSrSrg)KwargsForCausalLMirN)rGrHrIrJrMrr;r9rwrws3r;rwc^\rSrSrS/rSS0rSS/S/40rU4SjrSrS r S r S r S r S r \\SS\\R$S\\R&S\\R$S\\S\\R*S\\R$S\\S\\S\\S\\R$S\\\R&4S\\S\4Sjj55rSU4SjjrSrU=r$) Phi3ForCausalLMizlm_head.weightlm_head colwise_repr<logitsc>[TU]U5 [U5UlURUl[ R "URURSS9UlUR5 gNFr*) r,r-r#rr&rr/r0rzr/r6s r9r-Phi3ForCausalLM.__init__sU  v&  ++yy!3!3V5F5FUS  r;c.URR$rrr'rs r9r3$Phi3ForCausalLM.get_input_embeddingszz&&&r;c$XRlgrrr6s r9r7$Phi3ForCausalLM.set_input_embeddings "' r;cUR$rrzrs r9get_output_embeddings%Phi3ForCausalLM.get_output_embeddingss ||r;cXlgrr)r7new_embeddingss r9set_output_embeddings%Phi3ForCausalLM.set_output_embeddingss% r;cXlgrr)r7decoders r9 set_decoderPhi3ForCausalLM.set_decoders r;cUR$rrrs r9 get_decoderPhi3ForCausalLM.get_decoders zzr;r9rdrrr:labelsrrr;rlogits_to_keeprtr=c UbUOURRnU bU OURRn UR"SUUUUUUUU U S. U D6n U Rn[ U [ 5(a [U *S5OU nURUSS2USS245nSnUb)UR"SUX`RRS.U D6n[UUU RU RU RS9$)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, Phi3ForCausalLM >>> model = Phi3ForCausalLM.from_pretrained("meta-phi3/Phi3-2-7b-hf") >>> tokenizer = AutoTokenizer.from_pretrained("meta-phi3/Phi3-2-7b-hf") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```N) r9rdrrr:rrr;r)r|rr&lossr|rr<r@r)r.rr;rr?rrslicerz loss_functionr&rrr<r@)r7r9rdrrr:rrrr;rrrtrr< slice_indicesr|rs r9rEPhi3ForCausalLM.forwardsN2C1N-TXT_T_TqTq$8$D $++JjJj  ,0:: , )%+'/!5) ,  ,  11 8B>SV8W8W~ot4]k mA}a,?@A  %%pVF{{OeOepiopD%#33!//))   r;c >U(aeURR(aJURSURRS-:a USn XRR::aSn[T U]"SUUUUUUUUS.U D6n U $)Nr!r)r9rrdr:rrrrr)r.rrQ original_max_position_embeddingsr,prepare_inputs_for_generation) r7r9rrdr:rrrrrt past_length model_inputsr8s r9r-Phi3ForCausalLM.prepare_inputs_for_generations$  (("dkk&R&RUV&VV(+KkkJJJ"&w<  +)')%)    r;)rzrr&) NNNNNNNNNNr)NNNNNTN) rGrHrIrJ_tied_weights_keys_tp_plan_pp_planr-r3r7rrrrrrrrKrrr rLrrrrrwrrErrMrNrOs@r9ryrys*+=)H_-z:;H'(&151537+/59-1$(,0/35934G E,,-G !.G u//0 G "% G   1 12 G ))*G D>G $D>G 'tnG !!1!12G c5<</0G *+G  G G X%%r;rya The Phi3 Model transformer with a sequence classification head on top (linear layer). [`Phi3ForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$)Phi3ForSequenceClassificationi/c>[TU]U5 URUl[U5Ul[ R "URURSS9UlUR5 gr~) r,r- num_labelsr#rrr/r0scorer/r6s r9r-&Phi3ForSequenceClassification.__init__>sS   ++v& YYv114??O  r;c.URR$rrrs r9r32Phi3ForSequenceClassification.get_input_embeddingsGrr;c$XRlgrrr6s r9r72Phi3ForSequenceClassification.set_input_embeddingsJrr;r9rdrrr:rrrr;r=c URUUUUUUUU S9n U Rn URU 5n UbURSn OURSn URR cU S:wa [ S5eURR cSnOUbXRR :gRU R[R5n[R"URSU R[RS9nUU-RS5nO.Sn[RURR S35 U [R"XRS 9U4nSnUbUR#XUURS 9n[%UUU R&U R(U R*S 9$) e labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rdrrr:rrr;Nrr!z=Cannot handle batch sizes > 1 if no padding token is defined.r?)rriz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r>)r|r pooled_logitsr.r)rr?rrQr.r%rArrrrKint32rCargmaxrrr8rGrrrr<r@)r7r9rdrrr:rrrr;transformer_outputsr<r|rSlast_non_pad_token non_pad_mask token_indicesrrs r9rE%Phi3ForSequenceClassification.forwardMs*8!F!Fr!J !#    >>**+,ZZ  u||J}}MOaab   %%VR_hlhshs%tD/ /??-;;*55   r;)rrrrt)rGrHrIrJr-r3r7rrrrKrrr rLrrrErMrNrOs@r9rr/s'(151537+/59-1$(,0/3A E,,-A !.A u//0 A "% A   1 12 A ))*A D>A $D>A 'tnA  *A A r;rc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$)Phi3ForTokenClassificationic>[TU]U5 URUl[U5Ul[ USS5b UR nO[ USS5b URnOSn[R"U5Ul [R"URUR5Ul UR5 g)Nclassifier_dropouthidden_dropoutg?)r,r-rr#rrrrrrrfr/r0rr/)r7r.rr8s r9r-#Phi3ForTokenClassification.__init__s   ++v& 6/ 6 B!'!:!:  V-t 4 @!'!6!6 !$ zz"45 YYv1163D3DE  r;c.URR$rrrs r9r3/Phi3ForTokenClassification.get_input_embeddingsrr;c$XRlgrrr6s r9r7/Phi3ForTokenClassification.set_input_embeddingsrr;r9rdrrr:rrrr;r=c  URUUUUUUUU S9n U Rn URU 5n URU 5n Sn UbUR XUR 5n [ U U U RU RS9$)rrN)rr|r<r@) rr?rfrrr.rr<r@)r7r9rdrrr:rrrr;rsequence_outputr|rs r9rE"Phi3ForTokenClassification.forwards*,0:: )%+'/!5,6 , "33,,7O,  %%fdkkBD$!//))   r;)rfrrrrt)rGrHrIrJr-r3r7rrrrKrrr rLrrrErMrNrOs@r9rrs '(151537+/59-1$(,0/3* E,,-* !.* u//0 * "% *   1 12 * ))** D>* $D>* 'tn*  * * r;r)rr#ryrr)r)Nr!)HtypingrrrrrKr activationsr cache_utilsr r r r generationr integrationsrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrmodeling_rope_utilsrrmodeling_utilsrrprocessing_utilsrutilsrrrrr configuration_phi3r"!torch.nn.attention.flex_attentionr#integrations.flex_attentionr$ get_loggerrGrModuler&rVrrr_rrzrrrrrrr#rwryrr__all__rr;r9rs).43 !OO)7>B9 LF&hh* !!;J   H %)bii)$( UU\\ U# U%,, U&% II% <<% % << % U\\* %  %%4@H)BIIH)VY'J"))J(J(I1IX*/**:<"))P)?PPf  S $7S  S lC !4C C L r;