2heSSKJr SSKJrJr SSKJr SSKrSSKJ r SSK J r J r J r SSKJrJrJrJrJr SS KJrJr SS KJr S S KJrJrJrJrJrJr SS K J!r!J"r" SSK#J$r$J%r% SSK&J'r'J(r( SSK)J*r*J+r+ SSK,J-r- SSK.J/r/ SSK0J1r1 SSK2J3r3 SSK4J5r5 SSK6J7r7 SS/r8"SS\!5r9"SS\"5r:"SS5r;"SS5r<"SS 5r="S!S"5r>g)#) annotations)IterableOptional)LiteralN)_legacy_response)eval_list_paramseval_create_paramseval_update_params) NOT_GIVENBodyQueryHeadersNotGiven)maybe_transformasync_maybe_transform)cached_property)Runs AsyncRunsRunsWithRawResponseAsyncRunsWithRawResponseRunsWithStreamingResponseAsyncRunsWithStreamingResponse)SyncAPIResourceAsyncAPIResource)to_streamed_response_wrapper"async_to_streamed_response_wrapper)SyncCursorPageAsyncCursorPage)AsyncPaginatormake_request_options)EvalListResponse)EvalCreateResponse)EvalDeleteResponse)EvalUpdateResponse)EvalRetrieveResponse)MetadataEvals AsyncEvalsc f\rSrSr\SSj5r\SSj5r\SSj5r\\SSS\S.SSjjr SSS\S.SS jjr \\SSS\S.SS jjr \\\\SSS\S .SS jjr SSS\S.SS jjr Srg)r)%c,[UR5$N)r_clientselfs T/var/www/auris/envauris/lib/python3.13/site-packages/openai/resources/evals/evals.pyruns Evals.runs&sDLL!!c[U5$z This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content. For more information, see https://www.github.com/openai/openai-python#accessing-raw-response-data-eg-headers )EvalsWithRawResponser0s r2with_raw_responseEvals.with_raw_response*s$D))r5c[U5$z An alternative to `.with_raw_response` that doesn't eagerly read the response body. For more information, see https://www.github.com/openai/openai-python#with_streaming_response )EvalsWithStreamingResponser0s r2with_streaming_responseEvals.with_streaming_response4s*$//r5Nmetadataname extra_headers extra_query extra_bodytimeoutc xURS[UUUUS.[R5[ XVXxS9[ S9$) Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](https://platform.openai.com/docs/guides/evals). Args: data_source_config: The configuration for the data source used for the evaluation runs. Dictates the schema of the data used in the evaluation. testing_criteria: A list of graders for all eval runs in this group. Graders can reference variables in the data source using double curly braces notation, like `{{item.variable_name}}`. To reference the model's output, use the `sample` namespace (ie, `{{sample.output_text}}`). metadata: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. name: The name of the evaluation. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds /evalsdata_source_configtesting_criteriarArBrCrDrErFbodyoptionscast_to)_postrr EvalCreateParamsr"r$ r1rKrLrArBrCrDrErFs r2create Evals.create=sW`zz  *<(8 (  #33)+Q['  r5rMc nU(d[SU<35eURSU3[X#XES9[S9$) Get an evaluation by ID. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds 6Expected a non-empty value for `eval_id` but received /evals/rMrPrQ ValueError_getr"r'r1eval_idrCrDrErFs r2retrieveEvals.retrieve~sO.UV]U`ab byygY (+Q[)   r5c U(d[SU<35eURSU3[UUS.[R5[ XEXgS9[ S9$) Update certain properties of an evaluation. Args: metadata: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. name: Rename the evaluation. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rYrZrArBrMrN)r]rRrr EvalUpdateParamsr"r&r1r`rArBrCrDrErFs r2update Evals.updatespDUV]U`ab bzzgY  ( #33 )+Q['  r5afterlimitorderorder_byrCrDrErFcURS[[[UUUU[ UUUUS.[ R 5S9[S9$a List evaluations for a project. Args: after: Identifier for the last eval from the previous pagination request. limit: Number of evals to retrieve. order: Sort order for evals by timestamp. Use `asc` for ascending order or `desc` for descending order. order_by: Evals can be ordered by creation time or last updated time. Use `created_at` for creation time or `updated_at` for last updated time. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rI)rkrlrmrn)rCrDrErFquery)pagerPmodel) _get_api_listrr#r"rr EvalListParams r1rkrlrmrnrCrDrErFs r2list Evals.listsfH!!  01(+'%%!&!&!&$,  %33 #%"  r5c nU(d[SU<35eURSU3[X#XES9[S9$) Delete an evaluation. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rYrZrMr[r]_deleter"r%r_s r2delete Evals.delete sO.UV]U`ab b||gY (+Q['   r5)returnr)rr8)rr=rKz#eval_create_params.DataSourceConfigrLz-Iterable[eval_create_params.TestingCriterion]rAOptional[Metadata] | NotGivenrBstr | NotGivenrCHeaders | NonerD Query | NonerE Body | NonerF'float | httpx.Timeout | None | NotGivenrr$ r`strrCrrDrrErrFrrr'r`rrArrBrrCrrDrrErrFrrr&)rkrrlint | NotGivenrm!Literal['asc', 'desc'] | NotGivenrn.Literal['created_at', 'updated_at'] | NotGivenrCrrDrrErrFrrz SyncCursorPage[EvalListResponse] r`rrCrrDrrErrFrrr%__name__ __module__ __qualname____firstlineno__rr3r9r>r rUrarhrwr}__static_attributes__rr5r2r)r)%s;""**003<()-$("&;D? @? H ? 0 ?  ? &? "?  ? 9?  ? N)-$("&;D  &  "   9   J3<()-$("&;D1 1 0 1  1 &1 "1  1 91  1 l!* )3"AsyncEvals.with_streaming_response<s/t44r5Nr@c # URS[UUUUS.[R5IShvN [ XVXxS9[ S9IShvN $NN7f)rHrIrJNrMrN)rRrr rSr"r$rTs r2rUAsyncEvals.createEsn`ZZ ,*<(8 (  #33)+Q['     s!/AA A A AArMc # U(d[SU<35eURSU3[X#XES9[S9IShvN $N7f)rXrYrZrMr[Nr\r_s r2raAsyncEvals.retrieves\.UV]U`ab bYYgY (+Q[)     :AAAc # U(d[SU<35eURSU3[UUS.[R5IShvN [ XEXgS9[ S9IShvN $NN7f)rdrYrZreNrMrN)r]rRrr rfr"r&rgs r2rhAsyncEvals.updatesDUV]U`ab bZZgY , ( #33 )+Q['     s$AA)A% A) A'!A)'A)rjcURS[[[UUUU[ UUUUS.[ R 5S9[S9$rp)rtr r#r"rr rurvs r2rwAsyncEvals.listsfH!!  !12(+'%%!&!&!&$,  %33 #%"  r5c # U(d[SU<35eURSU3[X#XES9[S9IShvN $N7f)rzrYrZrMr[Nr{r_s r2r}AsyncEvals.deletes\.UV]U`ab b\\gY (+Q[' "    rr)rr)rr)rrrrr)rkrrlrrmrrnrrCrrDrrErrFrrzCAsyncPaginator[EvalListResponse, AsyncCursorPage[EvalListResponse]]rrrr5r2r*r*-s<''//553<()-$("&;D? @? H ? 0 ?  ? &? "?  ? 9?  ? N)-$("&;D  &  "   9   J3<()-$("&;D1 1 0 1  1 &1 "1  1 91  1 l!* )3> LL  )@@ NN  '>> LL  %<< JJ  '>> LL  r5c@[URR5$r.)rrr3r0s r2r3EvalsWithRawResponse.runsIs"4;;#3#344r5rrUr}rwrarhNrr)rNone)rrrrrrrrr3rrr5r2r8r85s &55r5r8c2\rSrSrSSjr\SSj5rSrg)riNcXl[R"UR5Ul[R"UR5Ul[R"UR 5Ul[R"UR 5Ul[R"UR5Ulgr.)rrasync_to_raw_response_wrapperrUrarhrwr}rs r2r"AsyncEvalsWithRawResponse.__init__Os &DD LL  )FF NN  'DD LL  %BB JJ  'DD LL  r5c@[URR5$r.)rrr3r0s r2r3AsyncEvalsWithRawResponse.runsbs' (8(899r5rNrr*rr)rrrrr5r2rrNs &::r5rc2\rSrSrSSjr\SSj5rSrg)r=igcXl[UR5Ul[UR5Ul[UR5Ul[UR 5Ul[UR 5Ulgr.)rrrUrarhrwr}rs r2r#EvalsWithStreamingResponse.__init__hsn 2 LL  5 NN  3 LL  1 JJ  3 LL  r5c@[URR5$r.)rrr3r0s r2r3EvalsWithStreamingResponse.runs{s()9)9::r5rNr)rrrrr5r2r=r=gs &;;r5r=c2\rSrSrSSjr\SSj5rSrg)ricXl[UR5Ul[UR5Ul[UR5Ul[UR 5Ul[UR 5Ulgr.)rrrUrarhrwr}rs r2r(AsyncEvalsWithStreamingResponse.__init__sn 8 LL  ; NN  9 LL  7 JJ  9 LL  r5c@[URR5$r.)rrr3r0s r2r3$AsyncEvalsWithStreamingResponse.runss-dkk.>.>??r5rNr)rrrrr5r2rrs &@@r5r)? __future__rtypingrrtyping_extensionsrhttpxrtypesr r r _typesr r rrr_utilsrr_compatr runs.runsrrrrrr _resourcerr _responserr paginationrr _base_clientr!r"types.eval_list_responser#types.eval_create_responser$types.eval_delete_responser%types.eval_update_responser&types.eval_retrieve_responser'types.shared_params.metadatar(__all__r)r*r8rr=rrr5r2rs#%% MM??<&;Y9@8<<<@4 L !E OE PE !E P552::2;;2@@r5