_Di~%SrSSKJr SSKrSSKrSSKJr SSKrSSK r SSK r SSK r SSK r SSK r SSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKJrJrJrJrJr SSK J!r! SSK"J#r# SSK$J%r% SSKJ&r&J'r'J(r(J)r)J*r*J+r+J,r,J-r- SS K.J/r0 SSK1r1SS K2J3r3 SS K1J4r5 SS K6J7r8 SS K9J:r:J;r; SSKJ?r? SSK@r@SSK@JArB SSK@JCrD SSK@JErF SSKGJHrH SSKIJJrJ SSKIJKrL SSKMJNrN SSKOJPrP SSKQJRrRJSrSJTrTJUrU SSKVJWrWJXrXJYrY SSKZJ[r[J\r\J]r]J^r^J_r_J`r`JaraJbrb SSKcJdre SSKfJgrg SSKhJiriJjrj SrkSrlSYS jrmS!rnSS"KoJprp SS%KrJsrs \&(aASSKtruSS(KvJwrw SSK@JCrC SS)KxJyrz SS*K{J|r| SS+K}J~r SS,KJrJrJr SS-KJrJrJrJrJrJrJrJr \GR"\5r\GR"S.5rS/rS0rS1\S2'S3\!"\=5GR(;rS4rSZS[S5jjrS\S6jr\,\GR2\4r\*S7r\ GR:"S8S99S]S:j5rS^S;jrS_S<jrS`S=jrSaSbS>jjrScS?jrSaSdS@jjr\RvSeSAj5r\RvSBSC.SfSDjj5rSESC.SgSFjjr\ GR:"S8S99SG5r"SHSI\5GRN5r"SJSK5rSaShSLjjrSaShSMjjr"SNSO\ GRX5rSSP.SiSQjjrSjSRjrSkSSjrSlSTjrSmSUjrSESV.SnSWjjrSoSXjrg!\qa "S#S$5rpGNf=f!\qa "S&S'5rsGNf=f!\qa \(rz\(r|GNf=f)paClient for interacting with the LangSmith API. Use the client to customize API keys / workspace connections, SSL certs, etc. for tracing. Also used to create, read, update, and delete LangSmith resources such as runs (~trace spans), datasets, examples (~records), feedback (~metrics), projects (tracer sessions/groups), etc. For detailed API documentation, visit the [LangSmith docs](https://docs.langchain.com/langsmith/home). ) annotationsN) AsyncIterableIterableIteratorMappingSequence) signature)Path) PriorityQueue) TYPE_CHECKING AnnotatedAnyCallableLiteralOptionalUnioncast)parse)Field)adapters) multipart) TypeGuardoverload)PoolKey)Retry)env)schemas)utils)_orjson)TracingQueueItem)tracing_control_thread_func) warn_beta)CompressedTraces)_AUTO_SCALE_UP_NTHREADS_LIMIT_BLOCKSIZE_BYTES _BOUNDARY_SIZE_LIMIT_BYTES) MultipartPartMultipartPartsAndContext join_multipart_parts_and_context)SerializedFeedbackOperationSerializedRunOperation#combine_serialized_queue_operations$compress_multipart_parts_and_contextserialize_feedback_dictserialize_run_dictcnSSKJn SSKJn SSKJn SSKJn XX#4$![a [S5ef=f)z,Dynamically import OTEL modules when needed.rtrace)set_span_in_context)get_otlp_tracer_provider OTELExporterzKTo use OTEL tracing, you must install it with `pip install langsmith[otel]`) opentelemetryrCopentelemetry.tracerD%langsmith._internal.otel._otel_clientrE'langsmith._internal.otel._otel_exporterrG ImportError) otel_tracerDrErGs r? _import_otelrNmsC  5;  I0HVV   Y   s4)ZoneInfoc\rSrSrSrSrg)rOzIntroduced in python 3.9.r=N__name__ __module__ __qualname____firstlineno____doc____static_attributes__r=r>r?rOrOs'r>rO)TracerProviderc\rSrSrSrSrg)rYzUsed for optional OTEL tracing.r=NrRr=r>r?rYrYs-r>rY)RunnablerBrF evaluator) AEVALUATOR_T ATARGET_TAsyncExperimentResults)COMPARATIVE_EVALUATOR_TDATA_T EVALUATOR_T EXPERIMENT_TSUMMARY_EVALUATOR_TTARGET_TComparativeExperimentResultsExperimentResultszurllib3.connectionpool x-api-keyr=ztuple[dict, ...] EMPTY_SEQ key_blocksizez:How are people using my agent? What are they asking about?c[U[R5(d[R"U5(a U[U54$[ R "[U55nURRS5n[U5U:aXR*n[USS9 O[R"SUSU35eURS:XaSnX4$URS:XaS nX4$![a Nf=f) z*Parse a public dataset URL or share token./z token parts)varzInvalid public z URL: zsmith.langchain.comhttps://api.smith.langchain.comzbeta.smith.langchain.comz$https://beta.api.smith.langchain.com) isinstanceuuidUUIDstr ValueError urllib_parseurlparsepathsplitlen_as_uuidr;LangSmithUserErrornetloc) url_or_tokenapi_url num_partskind parsed_url path_parts token_uuids r?_parse_token_or_urlrs lDII . .$))L2I2IC -- -3J &&s<'89J&&s+J :)# + /))OD6 ~*VWW113     8 88  !    sAC)) C65C6c[R"U5RRS5SnUS:H=(d UR S5$![ a gf=f)zCheck if the URL is langchain hosted. Args: url (str): The URL to check. Returns: bool: True if the URL is langchain hosted, False otherwise. :rz langchain.comz.langchain.comF)rvurlsplitr}ryendswith Exception)urlr}s r?_is_langchain_hostedrsY&&s+2288=a@(MFOO)>s)C DEP*. &'  " " 2\ 22r>cN[RS5 UR5 g)zPClose the session. Args: session (requests.Session): The session to close. zClosing Client.sessionN)loggerdebugclose)sessions r? close_sessionr s  LL)* MMOr>cU(dn[U5(a][R"S5(dA[R"5(a&[R "S[R 5 ggggg)zVerify API key is provided if url not localhost. Args: api_url (str): The API URL. api_key (Optional[str]): The API key. Returns: None Raises: LangSmithUserError: If the API key is not provided when using the hosted service. r:z8API key must be provided when using hosted LangSmith APIN)rr;r<tracing_is_enabledwarningswarnLangSmithMissingAPIKeyWarningrapi_keys r?_validate_api_key_if_hostedrs^   ) )..~>>++-- MMJ66 .? * r>cH[U[5(a [US5$U$)zFormat a feedback score by truncating numerical values to 4 decimal places. Args: score: The score to format, can be a number or any other type Returns: The formatted score )rqfloatround)scores r?_format_feedback_scorer/s"%UA Lr>cUc[R"S5nU(dgO [U5n[U5nUS:dUS:a[R"SU35eU$)zYGet the tracing sampling rate. Returns: Optional[float]: The tracing sampling rate. NTRACING_SAMPLING_RATErrzELANGSMITH_TRACING_SAMPLING_RATE must be between 0 and 1 if set. Got: )r; get_env_varrtrr|)tracing_sampling_ratesampling_rate_str sampling_rates r?_get_tracing_sampling_rater>sy$$001HI ! 56+,MqMA-)) "O %   r>cU=(d 0n0nUR5Hup#UR5nU(d[R"S5eUR5RS5RS5R S5nUR5RS5RS5n[ XE5 XQU'M U$)Nz&LangSmith runs API URL cannot be empty"'rn)itemsstripr;r|rstripr)_write_api_urlsprocessed_write_api_urlsrr processed_urlprocessed_api_keys r?_get_write_api_urlsrUs%+O!'--/  --.VW W%++-33C8>>sCJJ3O #MMO11#6<c[U[R5(d[R"U5$U$![a,nU=(d Sn[R "USU35UeSnAff=f)Nvaluez* must be a valid UUID or UUID string. Got )rqrrrsrur;r|)rroes r?r{r{esi'1%'C'CtyyNN nW))e=eW E  s499 A/'A**A/cgNr=)rs r? _ensure_uuidrosGJr>T accept_nullcgrr=rrs r?rrssr>FcBUcU(ag[5$[U5$r)r4r{rs r?rrys } w E?r>cn[R"U5nURRS5SnU$)Nrr)rvrwr}ry)rrhosts r? _parse_urlrs2&&s+J    " "3 ' *D Kr>c^\rSrSr/SQr\R \R \R\RS4SU4Sjjjr SU4Sjjr Sr U=r $) _LangSmithHttpAdapteri) max_retriesconfig_pool_connections _pool_maxsize _pool_block _blocksizei@c2>XPl[TU] XX45 gr)rsuper__init__)selfpool_connections pool_maxsizer pool_block blocksize __class__s r?r_LangSmithHttpAdapter.__init__s$ )Qr>c \>[(aURUS'[TU] "XU40UD6$)Nr)URLLIB3_SUPPORTS_BLOCKSIZErrinit_poolmanager)r connectionsrblock pool_kwargsrs r?r&_LangSmithHttpAdapter.init_poolmanagers. % %'+K $w' eS{SSr>)r) rrrrrzUnion[Retry, int, None]rboolrrreturnNoneF) rSrTrUrV __attrs__requests_adaptersDEFAULT_POOLSIZEDEFAULT_RETRIESDEFAULT_POOLBLOCKrrrX __classcell__rs@r?rrsI!2 B B->>/@/P/P,>> R R R- R  R  R  R RTTr>rc\rSrSr%Sr/SQrS\S'S\S'S\S'S \S 'S \S 'GS S S S S S SS S S S SS S S S S S S S S S S S S.GS SjjjrGSSjrGSSjr GSSjr \ GSSj5r \ GSSj5r GSSjrGSSjr\ GSSj5r\R"GSSj5r\ GSSj5r\R"GSSj5r\ GSSj5rGSSjrGSSjrS S S S S S!S".GSS#jjrS S$.GSS%jjrS S&S'S(.GSS)jjrS \R4R6S*.GSS+jjrS S \R4R6S,.GSS-jjrGSGSS.jjrGSS/jrGSS0jr SS1.GS S2jjr!S S SS S S3.GS!S4jjr"S S S5.GS"S6jjr#S S S5.GS#S7jjr$GS$S8jr%GS%S9jr&GS&S:jr'GSS;jr(GS'S<jr)S S S=.GS(S>jjr*GS)SS?.GS*S@jjjr+S S S=.GS+SAjjr,S S S=.GS,SBjjr-GS)SSSC.GS-SDjjjr.SES S SF.GS.SGjjr/SESH.GS/SIjjr0S S S S S S S S S S S SS S S SJ.GS0SKjjr1S S S5.GS1SLjjr2S S S5.GS2SMjjr3GS3GS4SNjjr4GS'SOjr5GS5SPjr6GS6GS7SQjjr7S S S S S S S S S S S S S S S S SR.GS8SSjjr8S S S S S S S S S S S S S S S S ST.GS9SUjjr9S S SV.GS:SWjjr:S SX.GS;SYjjr;GSS\jr>GS GS?S]jjr?GS GS@S^jjr@GS S S_.GSAS`jjjrAGS S S_.GSASajjjrBGSBSbjrCGSCScjrDS S Sd.GSDSejjrES S S S Sf.GSESgjjrFS S SS S Sh.GSFSijjrGS S S S S Sj.GSGSkjjrHGSHSljrIGSISmjrJ\KR"Sn5S S SSo.GSJSpjj5rMS Sq.GSKSrjjrNS S Sn.GSLSsjjrOGSMGSNStjjrP\KR"SV5S S SV.GSOSujj5rQS \R4R6S S S S Sv.GSPSwjjrRS S Sx.GSQSyjjrS\KR"Sx5S S Sx.GSRSzjj5rTGS S S_.GSSS{jjjrUGS S S_.GSTS|jjjrVS S S S S S S}.GSUS~jjrW\KR"S5S S S.GSVSjj5rXS S S.GSWSjjrYS S S S S.GSXSjjrZS S S S S.GSYSjjr[S S S.GSZSjjr\GS[Sjr]\KR"S5GS\GS]Sjj5r^\KR"S5GS\GS^Sjj5r_GS_GS`Sjjr`GSGSaSjjraS SS.GSbSjjrbS SS.GSbSjjrcS SS.GScSjjrdGSdSjreGSeGSfSjjrfS SS.GScSjjrgS SS.GSgSjjrh\KR"S5S S S SS S.GShSjj5riSrjSrk\KR"S5GSiGSjSjj5rlS S.GSkSjjrmGSlSS S S SS.GSmSjjjrn\oSS.GSnSjj5rp\oGSoSj5rq\oS S.GSpSjj5rrS S S S S S S S.GSqSjjrsS S S SS.GSrSjjrtGSsSjruSS.GStSjjrvS S S S.GSuSjjrwS S SS.GSvSjjrxGSwSjryGSxSjrzS S.GSySjjr{S S SS.GSzSjjr|GS_S S.GS{Sjjjr}S S SS.GS|Sjjr~GS}S S S S S S \RGRS S S SS S S S S S.GS~SjjjrS S S S S.GSSjjrGSSjrS S S S S.GSSjjrGSSjrGS S S S S S.GSSjjjrS S S S.GSSjjrS S S.GSSjjrS S.GSSjjrS S S SS.GSSjjrGSSjrS S S.GSSjjrGSSjrGSSjrS S S S S.GSSjjrS S S S.GSSjjrGSSjrS S S.GSSjjrGSSjrGSSjrGSSjrGSSjrS S S S S S.GSSjjrGSSjrGSSjrGSGSSjjrGSSjrGSSjrGSSjrGSSjrGSSjrSSS S\GR@GRBSS S.GSSjjrGSSjrS S S SS.GSSjjrS S.GSSjjrS S S S S S.GSSjjrGSSjrSS.GSSjjrS SSS.GSSjjrSS.GSSjjrS SS S S S S.GSSjjrGS'Sjr\GSGSSjj5r\GSGSSjj5rGSGSSjjrGSGSSjjrGSGSGSjjrGSGSGSjjr\o\S S S S GS.GSGSjj5r\oS S S GSGSSGS.GSGSjj5rS S S GS.GSGS jjrGSGS jrGS rg (Clientiz.Client for interacting with the LangSmith API.)) __weakref__r_api_key _workspace_id_headers_custom_headers retry_config timeout_ms_timeoutr_get_data_type_cached_web_url _tenant_idtracing_sample_rate_filtered_post_uuids tracing_queue _anonymizer _hide_inputs _hide_outputs_hide_metadata_omit_traced_runtime_info_process_buffered_run_ops_run_ops_buffer_size_run_ops_buffer_timeout_ms_run_ops_buffer_last_flush_time_infor _settings_manual_cleanup _pyo3_clientcompressed_traces_data_available_event_futures_run_ops_buffer_run_ops_buffer_lock otel_exporter _otel_trace_set_span_in_context_max_batch_size_bytes_tracing_error_callback_multipart_disabled Optional[str]r dict[str, str]rrztuple[float, float]rrr$NTF)rrrweb_urlrauto_batch_tracing anonymizer hide_inputs hide_outputs hide_metadataomit_traced_runtime_infoprocess_buffered_run_opsrun_ops_buffer_sizerun_ops_buffer_timeout_msinfoapi_urlsotel_tracer_provider otel_enabledr workspace_idmax_batch_size_bytesheaderstracing_error_callbackc U(aU(a[R"S5e[R"S5(d[R"S5(a1[R"S5(a[R"S5e[ U5Ul[ 5Ul[U5Ul [R"U5Ul U=(d 0Ul UR(aB[[UR55UlURURUlOs[R""U5Ul[R$"U5Ul['URUR 5 URUR 0Ul U=(d [)5Ul[-U[.5(aXD4O U=(d SUlUR0SS- UR0S S- 4UlXPlS UlU(aUO[8R:"5nUUlUb[-U[>R@5(aUO[>R@"S!0UD6Ul![DRF"U[HUR<5 [JRL"[HU5 S Ul'S Ul(S Ul)/Ul*[VRX"5Ul-S Ul.UUl/S Ul0U(aL[c5Ul2[VRf"[h[DRj"U54S 9Rm5 OS Ul2[oUR*[p[rS -S 9nUR<RuSU5 UR<RuSU5 [vRx"SS9"URz5Ul>Xl?U bU O[R"S5S:HUlAU bU O[R"S5S:HUlBU bU O[R"S5S:HUlCXlDXlEXlFU=(d SUlG[R"5UlIU bUc [S5eU cUb [S5eS UlK[R"S5bFS nSS KLnU(a8SnSnSnS n URURUR UUUU 5UlKS UlSS UlT[5(dU(a[5un!n"n#n$U!R"5n%U%R[5n&Uc[-U%U!R5(aJ[U&S5(a9[-[U!RU&5RU!R5(d[[U%5nOU#"5nU!R"U5 U$"US9Ul.U!UlbU"UlcOS Ul.UUlfg ![a*n[RS[U55 S nAGNS nAff=f![a*n[RS[U55 S nAGN{S nAff=f![a [R"S 5 S Ul.Nf=f)"a(Initialize a `Client` instance. Args: api_url (Optional[str]): URL for the LangSmith API. Defaults to the `LANGCHAIN_ENDPOINT` environment variable or `https://api.smith.langchain.com` if not set. api_key (Optional[str]): API key for the LangSmith API. Defaults to the `LANGCHAIN_API_KEY` environment variable. retry_config (Optional[Retry]): Retry configuration for the `HTTPAdapter`. timeout_ms (Optional[Union[int, Tuple[int, int]]]): Timeout for the `HTTPAdapter`. Can also be a 2-tuple of `(connect timeout, read timeout)` to set them separately. web_url (Optional[str]): URL for the LangSmith web app. Default is auto-inferred from the `ENDPOINT`. session (Optional[requests.Session]): The session to use for requests. If `None`, a new session will be created. auto_batch_tracing (bool, default=True): Whether to automatically batch tracing. anonymizer (Optional[Callable[[dict], dict]]): A function applied for masking serialized run inputs and outputs, before sending to the API. hide_inputs (Optional[Union[Callable[[dict], dict], bool]]): Whether to hide run inputs when tracing with this client. If `True`, hides the entire inputs. If a function, applied to all run inputs when creating runs. hide_outputs (Optional[Union[Callable[[dict], dict], bool]]): Whether to hide run outputs when tracing with this client. If `True`, hides the entire outputs. If a function, applied to all run outputs when creating runs. hide_metadata (Optional[Union[Callable[[dict], dict], bool]]): Whether to hide run metadata when tracing with this client. If `True`, hides the entire metadata. If a function, applied to all run metadata when creating runs. omit_traced_runtime_info (bool): Whether to omit runtime information from traced runs. If `True`, runtime information (SDK version, platform, Python version, etc.) will not be stored in the `extra.runtime` field of runs. Defaults to `False`. process_buffered_run_ops (Optional[Callable[[Sequence[dict]], Sequence[dict]]]): A function applied to buffered run operations that allows for modification of the raw run dicts before they are converted to multipart and compressed. Useful specifically for high throughput tracing where you need to apply a rate-limited API or other costly process to the runs before they are sent to the API. Note that the buffer will only flush automatically when `run_ops_buffer_size` is reached or a new run is added to the buffer after `run_ops_buffer_timeout_ms` has elapsed - it will not flush outside of these conditions unless you manually call `client.flush()`, so be sure to do this before your code exits. run_ops_buffer_size (Optional[int]): Maximum number of run operations to collect in the buffer before applying `process_buffered_run_ops` and sending to the API. Required when `process_buffered_run_ops` is provided. run_ops_buffer_timeout_ms (Optional[int]): Maximum time in milliseconds to wait before flushing the run ops buffer when new runs are added. Defaults to `5000`. Only used when `process_buffered_run_ops` is provided. info: The information about the LangSmith API. If not provided, it will be fetched from the API. api_urls (Optional[Dict[str, str]]): A dictionary of write API URLs and their corresponding API keys. Useful for multi-tenant setups. Data is only read from the first URL in the dictionary. However, ONLY Runs are written (`POST` and `PATCH`) to all URLs in the dictionary. Feedback, sessions, datasets, examples, annotation queues and evaluation results are only written to the first. otel_tracer_provider (Optional[TracerProvider]): Optional tracer provider for OpenTelemetry integration. If not provided, a LangSmith-specific tracer provider will be used. tracing_sampling_rate (Optional[float]): The sampling rate for tracing. If provided, overrides the `LANGCHAIN_TRACING_SAMPLING_RATE` environment variable. Should be a float between `0` and `1`, where `1` means trace everything and `0` means trace nothing. workspace_id (Optional[str]): The workspace ID. Required for org-scoped API keys. max_batch_size_bytes (Optional[int]): The maximum size of a batch of runs in bytes. If not provided, the default is set by the server. headers (Optional[Dict[str, str]]): Additional HTTP headers to include in all requests. These headers will be merged with the default headers (User-Agent, Accept, x-api-key, etc.). Custom headers will not override the default required headers. tracing_error_callback (Optional[Callable[[Exception], None]]): Optional callback function to handle errors. Called when exceptions occur during tracing operations. Raises: LangSmithUserError: If the API key is not provided when using the hosted service. LangSmithUserError: If both `api_url` and `api_urls` are provided. z-You cannot provide both api_url and api_urls.LANGSMITH_ENDPOINTLANGCHAIN_ENDPOINTLANGSMITH_RUNS_ENDPOINTSz]You cannot provide both LANGSMITH_ENDPOINT / LANGCHAIN_ENDPOINT and LANGSMITH_RUNS_ENDPOINTS.)i'i_rrNF)targetargs)rrrhttp://https:// r HIDE_INPUTStrue HIDE_OUTPUTS HIDE_METADATAizOrun_ops_buffer_size must be provided when process_buffered_run_ops is specifiedzOprocess_buffered_run_ops must be provided when run_ops_buffer_size is specifiedUSE_PYO3_CLIENTzaFailed to import `langsmith_pyo3` when PyO3 client was requested, falling back to Python impl: %s@Bdz|Failed to instantiate `langsmith_pyo3.BlockingTracingClient` when PyO3 client was requested, falling back to Python impl: %s_tracer)tracer_providerzuLANGSMITH_OTEL_ENABLED is set but OpenTelemetry packages are not installed: Install with `pip install langsmith[otel]r=)gr;r|osgetenvrrsetrrrget_workspace_idr rnextiterrr get_api_url get_api_keyrrrrqrrrrrrequestsSessionr ls_schemas LangSmithInfor"weakreffinalizeratexitregisterr&r'r(r) threadingLockr*r+r.r0r rThread_tracing_control_thread_funcrefstartrr%r$mount functools lru_cache_get_data_typerrrrrrrrrr timer!rur%langsmith_pyo3rLrwarningreprBlockingTracingClientrr#r$r@rNget_tracer_provider get_tracerrSProxyTracerProviderhasattrr ProxyTracerrV NoOpTracerrYset_tracer_providerr,r-rrr/)'rrrrrr3rr4r5r6r7r8r9r:r;r<r=r>r?r@rrArBrCrDsession_adapterrsrqueue_capacity batch_sizebatch_timeout_millisworker_threadsrMrDrErGexisting_providertracers' r?rClient.__init__s x x--?  II* + +ryy9M/N/Nii233--0  $>>S#T 47E!B 79$-NN$4!59%9").  :G/D    3kk$')   eg!%D ())&7:   9g. :w/%.%8%8%D   & "&& %%m4> ' %%n5? ( %%o6&@  *B&)A&$7!*C*Kt'/3yy{, $ /4G4Oa  $ +0C0Oa  !    1 2 >!N %!* '+$!"(6(L(L  &",& )D%EI$  L& * !N ', %/$B$B$D!*55h?'/##4j6T6TUU#FI66& * 6 6 &&g&11 04NDU/V,/G/I,"667KL%1BV%W"$. ,?)"&D '=$[ 6G .!NNZQ\ * L&*"  *sCX3//Y*C Z!3 Y'=Y""Y'* Z4ZZ!'[  [ c&URnSUS3$)zReturn an HTML representation of the instance with a link to the URL. Returns: str: The HTML representation of the instance. z LangSmith Client) _host_url)rlinks r? _repr_html_Client._repr_html_ s ~~4& VWWr>cUR(aURU5 gg![a [RSSS9 gf=f)zInvoke the background tracing error callback if configured. Args: error: The exception that occurred during background tracing. z!Error in tracing_error_callback: Texc_infoN)r/rrerror)rrs r?_invoke_tracing_error_callback%Client._invoke_tracing_error_callback)sM  ' ' ,,U3 (  8! s&AAc"SURS3$)zReturn a string representation of the instance with a link to the URL. Returns: str: The string representation of the instance. zClient (API URL: )rrs r?__repr__Client.__repr__8s #4<<.22r>c,[UR5$r)rrrs r?_host Client._host@s$,,''r>cX[R"URUR5$)zThe web host url.)r; get_host_urlrrrs r?rClient._host_urlDs$$T]]DLLAAr>cS[R3SS.nURUR5 UR(aURU[ 'UR (aUR US'U$)Nz langsmith-py/application/json)z User-AgentAcceptz X-Tenant-Id) langsmith __version__updaterr X_API_KEYr )rrCs r?_compute_headersClient._compute_headersIsd))*?*?)@A(  t++, <cz[RXU5 [RUSUR55 g)z8Set attributes that affect headers and recalculate them.rN)object __setattr__r)r attr_namers r?_set_header_affecting_attr!Client._set_header_affecting_attrWs-4E24T-B-B-DEr>cUR$)z+Return the API key used for authentication.)r rs r?rClient.api_key\s}}r>c(URSU5 g)Nr rrrs r?rras '' E:r>cUR$)z.Return the workspace ID used for API requests.)r rs r?rAClient.workspace_ides!!!r>c(URSU5 g)Nr rrs r?rArjs ''?r>c URb UR$[R"S5=(a [R"S5nU(a&[R"5UlUR$UR SSSS0UR S9n[R"U5 [R"S 0UR5D6UlUR$![a]n[RS URS [U535 [R"5UlSnAUR$SnAff=f) zGet the information about the LangSmith API. Returns: The information about the LangSmith API, or `None` if the API is not available. Nr: OTEL_ONLYGETz/inforr)rCtimeoutzFailed to get info from : r=)r"r;r<rbrcrequest_with_retriesrraise_for_status_with_textjson BaseExceptionrrtrru)rotel_only_moderesponsers r?r= Client.infons :: !:: "33  6((5  #113DJ::  400!#56 1H  / / 9#11DHMMODDJzz  4 NN*4<<.47)D $113DJJzz  4s;AC$$ E .AEE cURcQURSS5n[R"U5 [R "S0UR 5D6UlUR$)zbGet the settings for the current tenant. Returns: dict: The settings for the current tenant. rz /settingsr=)r#rr;rrbLangSmithSettingsr)rrs r? _get_settingsClient._get_settingssS >> !00 DH  / / 9'99LHMMOLDN~~r>cUb URcg[[RUR5nURnU(dgUR =(d UR S5nUcgX:a SUSUS3$g)Nsize_limit_byteszThe content length of z) bytes exceeds the maximum size limit of z bytes.)r"rrbrcbatch_ingest_configr.get)rcontent_lengthr=bic size_limits r?_content_above_sizeClient._content_above_sizes  !TZZ%7J,,djj9&&//N377;M3N    &((89))3 G= r>r)request_kwargsstop_after_attemptretry_on to_ignorehandle_response_contextc U=(d 0nSUR0UEU ES0UREURS05EU RS05E0EnUS:wa-SU;a'SU;a!USRS5(dSUSS'[R"5[R "[ UR5S9/n /U=(d S Q[R[R[R4Q7n /U=(d S Q7n S n [U5Hun[R"[U 5 URR"U[!UR"U54S S 0UD6n S S S 5 [R$"U 5 U s $ [RL"S%USUS:35e!,(df  NG=f![&R(R*a`n[,R/S U5 US-U:XaeSU-[0R0"5S--n[2R4"U5 S nAGM S nAf[&R6GaPnU GbUb"US-U:aU"XS-5nU(aS nAGMaU R8S:Xa*[R"SUSUS[;U5U35eU R8S:Xa[R"SUUSU35eU R8S:Xa'[R<"SUS[;U5U35eU R8S:Xa'[R>"SUS[;U5U35eU R8S:Xa'[R@"SUS[;U5U35eU R8S:Xa'[RB"SUS[;U5U35eU R8S :Xa~U RE5nURS!S"5nUS#:Xa[RF"S$5eO![H[J4a Of=f[RL"S%USUS[;U535e[RL"S%USUS[;U535e[RF"S%USUS[;U535eS nAf[&RNGa3nUR"S&:waS'OS(n[QUR(a.[ URRRRS)55OS"5nURUU5nU(aUnO![Ha S nOf=fUR(a.URRRRS*5=(d S"OS"nUS S+US,S nnS-[WS.[YU5S/- 5-nUUU3n[R"S0USUSUS[;U5S1US2UU3 5UeS nAf[Zan[]UR^5n[YU5S:aUSOS"nURaS3S45nU(aZS5Rc[ US.5/U/-[YU5S:aUSS O/Vs/sHn[ U5PM Os snfsn-5nOUn[RL"S%USUSUU35UeS nAff=f!U a.nU b[,R/S U5 U sS nAs $S nAGMS nAf[R<a US-U:XaeU b_[eU RRRS6S755n O8![Za+n[,RgS8[;U55 S9n S nAOS nAff=fW SU--[0R0"5-n [2R4"U 5 GMU aC US-U:XaeSU-[0R0"5S--n[2R4"U5 GMf=f);a0Send a request with retries. Args: method (str): The HTTP request method. pathname (str): The pathname of the request URL. Will be appended to the API URL. request_kwargs (Mapping): Additional request parameters. stop_after_attempt (int, default=1): The number of attempts to make. retry_on (Optional[Sequence[Type[BaseException]]]): The exceptions to retry on. In addition to: `[LangSmithConnectionError, LangSmithAPIError]`. to_ignore (Optional[Sequence[Type[BaseException]]]): The exceptions to ignore / pass on. handle_response (Optional[Callable[[requests.Response, int], Any]]): A function to handle the response and return whether to continue retrying. _context (str, default=""): The context of the request. **kwargs (Any): Additional keyword arguments to pass to the request. Returns: The response object. Raises: LangSmithAPIError: If a server error occurs. LangSmithUserError: If the request fails. LangSmithConnectionError: If a connection error occurs. LangSmithError: If the request fails. rrCrdatafiles Content-Typer)rr=NstreamFzPassing on exception %srrizServer error caused failure to  z in LangSmith API. rz(Client took too long to send request to izRate limit exceeded for z. izAuthentication failed for izResource not found for iz Conflict for irr!org_scoped_key_requires_workspacezThis API key is org-scoped and requires workspace specification. Please provide 'workspace_id' parameter, or set LANGSMITH_WORKSPACE_ID environment variable.z Failed to rpz'Please confirm your LANGCHAIN_ENDPOINT.z(Please confirm your internet connection.zContent-Lengthrj*rz#Connection error caused failure to z Content-Length: z API Key: rzsession (project) z retry-after30zInvalid retry-after header: %sz in LangSmith API.)4rrrr;FilterLangSmithRetryFilterPoolFullWarningrtrLangSmithConnectionErrorLangSmithRequestTimeoutLangSmithAPIErrorrange filter_logs_urllib3_loggerrrequest_construct_urlrrr` exceptions ReadTimeoutrrrandomrrsleep HTTPError status_coderuLangSmithRateLimitErrorLangSmithAuthErrorLangSmithNotFoundErrorLangSmithConflictErrorrr|ruKeyErrorLangSmithErrorConnectionErrorrrCrmaxrzrlistrKreplacejoinrrt)!rmethodpathnamerrrrrrkwargslogging_filters retry_on_ to_ignore_ridxr sleep_timeshould_continue error_data error_coderecommendationrsize_recrprefixsuffixfillermasked_api_keyrKmsgargemsg retry_afters! r?rClient.request_with_retriessEL(-2 t}}       -- $$Y3**Y+   eO.(~-"9-11.AA8JN9 %n 5  ) ) +  * *DJJ @ 6 n"6 1100**6 7Lb6K +,C] B!--oO#'<<#7#7"*4<<B$$)$- $P77A#O-B%%(+= >  }PO **66LL!:A>Qw"44!"C6==?S+@!AJJJz*))?+*6"Qw);;2A(RSG2T#2$,#//36"*"<"<"A&"#H:.3377)#+*!.# &11S8"*"B"B"J6(#+*AhZ!9#&11S8"*"B"B":8*BtAwi#+*!.#&11S8"*"="="#O(33 (700')&+H,<,<,@,@PT,U&V $)< G') )*AsF2V]]_D  ;'' 700Vv}}'<=  :&  sG95F/.G/ F= 9GZ6AH92Z99Z6"R3Z9:ER>P RP!R P!!A;RZ64W" A$T.-W". T=:W"<T==B%W"" Z6/A9Z1(Y>=4Z11Z66Z99_+?[(_+($_+ %\32_+3 ](=!]#_+#]((7_+#A_+*_+paramsc#D# U(aUR5O0nURSS5nURSS5US'XCS'URSUUS9nUR5nU(dgUShvN [ U5US:agU[ U5- nMZN'7f)zGet a paginated list of items. Args: path (str): The path of the request URL. params (Optional[dict]): The query parameters. Yields: The items in the paginated list. offsetrlimitrUrrN)copyrrrrz)rrxrparams_rrrs r?_get_paginated_listClient._get_paginated_lists$*&++-rXq)";;w4 &H 001H MMOE  5zGG,, c%j F sA4B 6B7(B POSTruns)bodyrequest_methoddata_keyc#d# U(aUR5O0nURUUS[U50S9nUR5nU(dgUR U5(dgXtShvN UR S5nU(dgUR S5(dgUSUS'MN>7f)acGet a cursor paginated list of items. Args: path (str): The path of the request URL. body (Optional[dict]): The query body. request_method (Literal["GET", "POST"], default="POST"): The HTTP request method. data_key (str, default="runs"): The key in the response body that contains the items. Yields: The items in the paginated list. rrNcursorsr\cursor)rr _dumps_jsonrr) rrxr r!r"rr response_bodyr%s r?_get_cursor_paginated_list!Client._get_cursor_paginated_lists&"&$))+200K0 1H%MMOM  $$X..$. . .#'' 2G;;v&& 'GH ' /sA-B0/B.0?B0) description data_typec [R"5nURUSS9 URS5 UR SU4UUUUUS9$)aSUpload a dataframe as individual examples to the LangSmith API. Args: df (pd.DataFrame): The dataframe to upload. name (str): The name of the dataset. input_keys (Sequence[str]): The input keys. output_keys (Sequence[str]): The output keys. description (Optional[str]): The description of the dataset. data_type (Optional[DataType]): The data type of the dataset. Returns: Dataset: The uploaded dataset. Raises: ValueError: If the `csv_file` is not a `str` or `tuple`. Example: ```python from langsmith import Client import os import pandas as pd client = Client() df = pd.read_parquet("path/to/your/myfile.parquet") input_keys = ["column1", "column2"] # replace with your input column names output_keys = ["output1", "output2"] # replace with your output column names dataset = client.upload_dataframe( df=df, input_keys=input_keys, output_keys=output_keys, name="My Parquet Dataset", description="Dataset created from a parquet file", data_type="kv", # The default ) ``` F)indexrzdata.csv) input_keys output_keysr+namer,)ioBytesIOto_csvseek upload_csv)rdfr1r/r0r+r,csv_files r?upload_dataframeClient.upload_dataframesY`::< (% ( a  "!##   r>)r1r+r,cUUS.nU(aXGS'U(aXWS'U(a[R"U5US'[[R"55US'[ U[5(a,[ US5nSU0n URSS UU S 9n S S S 5 O5[ U[5(aURSS USU0S 9n O [S 5e[R"W 5 U R5n S U ;aHSU S ;a?[ U[5(aUOUSn U RS5Sn [SU S35e[R"S0U DURUR!5S.D6$!,(df  N=f)aUpload a CSV file to the LangSmith API. Args: csv_file (Union[str, Tuple[str, io.BytesIO]]): The CSV file to upload. If a string, it should be the path. If a tuple, it should be a tuple containing the filename and a `BytesIO` object. input_keys (Sequence[str]): The input keys. output_keys (Sequence[str]): The output keys. name (Optional[str]): The name of the dataset. description (Optional[str]): The description of the dataset. data_type (Optional[ls_schemas.DataType]): The data type of the dataset. Returns: Dataset: The uploaded dataset. Raises: ValueError: If the `csv_file` is not a string or tuple. Example: ```python from langsmith import Client import os client = Client() csv_file = "path/to/your/myfile.csv" input_keys = ["column1", "column2"] # replace with your input column names output_keys = ["output1", "output2"] # replace with your output column names dataset = client.upload_csv( csv_file=csv_file, input_keys=input_keys, output_keys=output_keys, name="My CSV Dataset", description="Dataset created from a CSV file", data_type="kv", # The default ) ``` )r/r0r1r+r,idrbfilerz/datasets/upload)rrNz"csv_file must be a string or tupledetailzalready existsrrnDataset z already existsrrr=)r;get_enum_valuertrruuid4rqopenrrrurrryrbDatasetr_get_optional_tenant_id) rr8r/r0r1r+r,rffile_rresult file_names r?r6Client.upload_csv!sj%&  L "-   ( 7 7 BD &T h $ $h% 44& 5&%% ( (00"x( 1HAB B++H5 v "2fX6F"F$.x$=$=8A;I!,R0Ix {/BC C!!  nn335  1&%s >E:: Fc[US5(a+[[US55(aUR5nO[ [U5nSU;a[ R "5US'O4[US[5(a[ R"US5US'SU;a@USb:U(a[R"US5US'URUS5US'SU;a@USb:U(a[R"US5US'URUS5US'SU;ac[US[5(aKUSnSU;a@USb:U(a[R"US5US'URUS5US'U(dPURS5(d:[ R R#[ R$R&5US'S U;aTURS 5S ;aUR)S S5 U$URS 5(aUS R)S S5 U$) aZTransform the given run object into a dictionary representation. Args: run (Union[ls_schemas.Run, dict]): The run object to transform. update (Optional[bool]): Whether the payload is for an "update" event. copy (Optional[bool]): Whether to deepcopy run inputs/outputs. Returns: dict: The transformed run object as a dictionary. rr<inputsNoutputsextrar start_time serializedrun_type)rrgraph)rzcallablegetattrrrrrrDrqrtrsr; deepish_copy_hide_run_inputs_hide_run_outputs_hide_run_metadatardatetimenowtimezoneutcpop)rrunrr run_createrPs r?_run_transformClient._run_transforms 3  HWS&-A$B$B"xxzJdCJ z !#zz|Jt   4(# . .#yyD)9:Jt  z !j&:&F'/'<'S(T 9%$($:$::i;P$QJy ! j Z 70CT%J%Jw'EU"uZ'8'D(0(=(=eJ>O(PE*%$($;$;E*c UR(ag[R"5nUHn[[UR S055nUR S05n0UEUEUS'UR S05n[R "5nURUR5VV s0sHupX;dM X_M sn n5 M gs sn nf)NrPruntimer) rls_envget_runtime_environmentrr setdefaultget_langchain_env_var_metadatarr) rr runtime_envra run_extrarerlangchain_metadatakvs r?_insert_runtime_envClient._insert_runtime_envs  ) ) 446 JT:#8#8"#EFI%00B?G#=k#=W#=Ii &11*bAH!'!F!F!H  OO"4":":"<R"<$!@Q"<R Ss - C <C cbURcg[R"5UR:$)NT)rrrs r?_should_sampleClient._should_samples(  # # +}}!9!999r>patchc:URc [U5$U(ac/nUHYn[US5nXPR;aUR U5 M3USU:XdM>URR U5 M[ U$/nUHnUR S5=(d USnXPR;aM1USU:XaEUR5(aUR U5 MbURRU5 MUR U5 M U$)Ntrace_idr<) rrr{rappendremoverrradd)rrrusampledr`rws r?_filter_for_samplingClient._filter_for_samplings  # # +:  G#C O4#<#<<NN3'Y(*--44X>  NG77:.;#d)888t9(**,,s+1155h?NN3' Nr>) project_name revision_iddangerously_allow_filesystemrrc $U=(d% U RS[R"55n0U EUUUUS.En URU /5(dgUb XZSSS'UR U SS9n UR U /5 U R S 5b]U S R5HFn [U [5(dM[U S [5(dM4U(aM=[S 5e UR(aoU R S 5(dYUR URRS U 45 UR!5(aUR#5 SSS5 gUR%XUS9 g!,(df  g=f)aPersist a run to the LangSmith API. Args: name (str): The name of the run. inputs (Dict[str, Any]): The input values for the run. run_type (str): The type of the run, such as tool, chain, llm, retriever, embedding, prompt, or parser. project_name (Optional[str]): The project name of the run. revision_id (Optional[Union[UUID, str]]): The revision ID of the run. api_key (Optional[str]): The API key to use for this specific run. api_url (Optional[str]): The API URL to use for this specific run. **kwargs (Any): Additional keyword arguments. Returns: None Raises: LangSmithUserError: If the API key is not provided when using the hosted service. Example: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() run_id = uuid4() client.create_run( id=run_id, project_name=project_name, name="test_run", run_type="llm", inputs={"prompt": "hello world"}, outputs={"generation": "hi there"}, start_time=datetime.datetime.now(datetime.timezone.utc), end_time=datetime.datetime.now(datetime.timezone.utc), hide_inputs=True, hide_outputs=True, ) ``` session_name)rr1rNrSNrPrrFr attachmentsrUMust set dangerously_allow_filesystem=True to allow passing in Paths for attachments.is_run_ops_buffer_flushpostrr)r_r;get_tracer_projectr|rbrorvaluesrqrr rurr*r)rx_should_flush_run_ops_buffer_flush_run_ops_buffer _create_run) rr1rNrSr~rrrrrra attachments r? create_runClient.create_runszn$ vzz   ' ' )(   (  ((*66   "=Hw  +M :((%(@    *. >>- ( 4(7>>@ z511":a=$7788$o A  ) )&**=V2W2W**$$++VZ,@A4466..0 +*   Z'  J+*s #AF Frc URS5GbURS5Gb URbURRU5 gURbUcUcURc [ S5e[ SU5n[U5unn[RSSUR5 URR [UUR[5nU(a9UR=RS- slURR5 SSS5 [![#UR%555 gUR&b[ SU5n[RSSUR(UR*5 UR,bQUR&R/[1USUUUUR3UR4R755S 95 gUR&R/[1USUUUS 95 gUR9XUS 9 gUR9XUS 9 g!,(df  GN =f) Nrw dotted_order@Run compression is enabled but threading event is not configuredrr5Adding compressed multipart to queue with context: %sr/Adding to tracing queue: trace_id=%s, run_id=%srr otel_contextr)rr%rr&r'rur0r2rlogcontextlockr.r& trace_countrZ _close_filesrrrrwr<r+putr r-r,get_current_span_create_run_non_batch)rrarr serialized_opmultipart_form opened_filesenqueueds r?rClient._create_runKs& NN: & 2~.:  ,!!,,Z8&&2OO--5$Z!36: F L!"  K"** ++00C&..! H  ..::a?:226681T,"5"5"789##/ 26: F  E!**!$$  %%1&&**(&~6)$+$+)-)B)B $ 0 0 A A C*  &&**(&~6)$+$+ **:PW*X  & &zG & T[10s AI Ic/nUcUbrU=(d URnU=(d URn0URE[U0EnUR SUS3[ U5US.[ R4S9 OjURR5HLup0URE[U 0EnUR SU S3[ U5US.[ R4S9 MN U(aO[U5S:a'[ R"US9n URU 5 U eURUS5 USeg![anURU5 SnANwSnAff=f![anURU5 SnAMSnAff=f)Nr/runsrrC)rrr)rr)rrrrrr'r;rrrxrrrzLangSmithExceptionGroupr) rrarrerrorstarget_api_urltarget_api_keyrCr write_api_url write_api_keyexception_groups r?rClient._create_run_non_batchs  '"5$4 N$4 NBB >BG !))%&e, +J 7#*$ (>>@*150D0D0J0J0L, ET]]EI}E %--(//$/ $;'.($,#B#B"D.1M 6{Q"*"B"Bf"U33OD%%33F1I>Qi % ! a   ! !%MM!$$%s0 0D<10E$< E!EE!$ F .FF cURSLa0$UR(a0[R"[ U55nURU5$URSLaU$URU5$NTF)rrrloadsr')rrN json_inputss r?rXClient._hide_run_inputssf    $I   !-- F(;cURSLa0$UR(a0[R"[ U55nURU5$URSLaU$URU5$r)rrrrr')rrO json_outputss r?rYClient._hide_run_outputssf    %I   "==W)=>L##L1 1    &N!!'**r>chURSLa0$URSLaU$URU5$r)r)rrs r?rZClient._hide_run_metadatas9   $ &I   % 'O""8,,r>cUR(dgURb$[UR5UR:agURb5[R"5UR - nXRS- :agg)zDCheck if the run ops buffer should be flushed based on size or time.FTrI)r)rrzr rrr!)rtime_since_last_flushs r?r#Client._should_flush_run_ops_bufferst##  % % 1D(()T-F-FF  * * 6$(IIK$2V2V$V !$)H)H4)OPr>crUR(dg[UR5nURR5 [R"5UlSSKJnJn URX0U5nURbURRU5 gg![a U"X5 gf=f)z8Process and flush run ops buffer in a background thread.Nr)LANGSMITH_CLIENT_THREAD_POOL_process_buffered_run_ops_batch) r)rclearrrr!&langsmith._internal._background_threadrrsubmitr(rz RuntimeError)rbatch_to_processrrfutures r?rClient._flush_run_ops_buffers##   4 45 ""$/3yy{, D188/7GF}}( !!&)) D +D C Ds%:B!!B65B6rc //S.nUGH4n[U[5(Ga[R"UR5nUR (a#[R "UR 5US'UR(a#[R "UR5US'UR(a#[R "UR5US'UR(a#[R "UR5US'UR(a#[R "UR5US'UR(a#[R "UR5US'UR(a[RS5 XERR!S UR"S UR$3[R&"U545 GM[U[(5(a[RS 5 GM[RS [+U55 GM7 UR,nUR.=(d1 UR0=(d 0R3S 5=(d [4n[6R8"[:5n [6R8"[:5n Sn [=[:[>SSS/5GHn [6R@"XL5n U (dM%U S:awU [CU SS5-U:a_URE[R&"U 5SU SSRGX53UUS9 Sn U RI5 U RI5 U RK5upU [CU5- n XR![R "U55 XR!U5 U (aMGM U (aPSRGSU RM555nURE[R&"U 5SU-UUS9 gg)N)rrurNrOeventsrPrrRzBAttachments are not supported when use_multipart_endpoint is Falseztrace=z,id=z;Feedback operations are not supported in non-multipart modez&Unknown item type in tracing queue: %srrrrurrr; )rrrc3R# UHupUSSRU53v M g7f)rrN)r).0rmrns r? /Client._batch_ingest_run_ops..gs'WCV411#R ! ~ 6CVs%')'rqr,rr_nonerNFragmentrOrrPrrRrrrt operationrxrwr<dumpsr+typer=r.rrr' collections defaultdictrrrdequerz_post_batch_ingest_runsrrpopleftr)ropsrrids_and_partial_bodyop curr_dictr=r body_chunks context_ids body_sizekey body_dequecurr_id curr_bodyrs r?_batch_ingest_run_opsClient._batch_ingest_run_opssW  B"455#MM"((3 99*1*:*:299*EIh'::+2+;+;BJJ+GIi(99*1*:*:299*EIh'88)0)9)9"(()CIg&88)0)9)9"(()CIg&==.5.>.>r}}.MIl+>>NN#%\\299bkk]$ruug6 i8PQB ;<<Q EtBxP9>yy  & & !((.B334FG !   ;F:Q:QRV:W :E:Q:QRV:W  W_568IJC$**+?+DEJ*M!C 1 a(8$99) pre_sampledc  U(dU(dgU=(d [Vs/sHo@RUSS9PM nnU=(d [Vs/sHnURUSSS9PM nnUHEnURS5(aURS5(aM1[R"S5e UHEnURS5(aURS5(aM1[R"S5e U(d!UR U5nUR USS 9nU(dU(dgUR (aBU(a[UR U55nU(a[UR U55nURXV-5 [[[[[[R"S U5S U55555nURU5 gs snfs snf) a Batch ingest/upsert multiple runs in the Langsmith system. Args: create (Optional[Sequence[Union[Run, RunLikeDict]]]): A sequence of `Run` objects or equivalent dictionaries representing runs to be created / posted. update (Optional[Sequence[Union[Run, RunLikeDict]]]): A sequence of `Run` objects or equivalent dictionaries representing runs that have already been created and should be updated / patched. pre_sampled (bool, default=False): Whether the runs have already been subject to sampling, and therefore should not be sampled again. Raises: LangsmithAPIError: If there is an error in the API request. Returns: None !!! note The run objects MUST contain the `dotted_order` and `trace_id` fields to be accepted by the API. Example: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() _session = "__test_batch_ingest_runs" trace_id = uuid4() trace_id_2 = uuid4() run_id_2 = uuid4() current_time = datetime.datetime.now(datetime.timezone.utc).strftime( "%Y%m%dT%H%M%S%fZ" ) later_time = ( datetime.datetime.now(datetime.timezone.utc) + timedelta(seconds=1) ).strftime("%Y%m%dT%H%M%S%fZ") runs_to_create = [ { "id": str(trace_id), "session_name": _session, "name": "run 1", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id)}", "trace_id": str(trace_id), "inputs": {"input1": 1, "input2": 2}, "outputs": {"output1": 3, "output2": 4}, }, { "id": str(trace_id_2), "session_name": _session, "name": "run 3", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id_2)}", "trace_id": str(trace_id_2), "inputs": {"input1": 1, "input2": 2}, "error": "error", }, { "id": str(run_id_2), "session_name": _session, "name": "run 2", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id)}." f"{later_time}{str(run_id_2)}", "trace_id": str(trace_id), "parent_run_id": str(trace_id), "inputs": {"input1": 5, "input2": 6}, }, ] runs_to_update = [ { "id": str(run_id_2), "dotted_order": f"{current_time}{str(trace_id)}." f"{later_time}{str(run_id_2)}", "trace_id": str(trace_id), "parent_run_id": str(trace_id), "outputs": {"output1": 4, "output2": 5}, }, ] client.batch_ingest_runs(create=runs_to_create, update=runs_to_update) ``` NFrT)rrrwrz:Batch ingest requires trace_id and dotted_order to be set.rtc3:# UHn[SU5v M g7frNr0rr`s r?r+Client.batch_ingest_runs..sQLS+FC88Lc3:# UHn[SU5v M g7fruNrrs r?rrsR\c+GS99\r)rkrbrr;r|r|rrrorr,r- itertoolsrr)rcreaterrr` create_dicts update_dictsserialized_opss r?batch_ingest_runsClient.batch_ingest_runsosFf =C2_  s G; Hc |UbUbX40nO URnUR5H^upg[RSUSU35 UR SUS3U0UR E[ U0ES.[R4SUS9 M` g![an[R"[U5U5n SRU 5R5n [RS U 35 O1![a$ [RS [!U535 Of=fUR#U5 SnAGMSnAff=f) Nz Sending batch ingest request to  with context: rz /runs/batchrr)rrrrrzFailed to batch ingest runs: )rrrrrrrr;rr tracebackformat_exception_onlyrrrrtrur) rr rrr endpointsrrrexc_desc_linesexc_descs r?rClient._post_batch_ingest_runssH  7#6 *I,,I.7oo.? *N 7 6~6FoV^U_`))%&k2 $$"mm$%~$$ (>>@'(%*  /@& 7N%.%D%DT!Wa%PN!ww~6==?HNN%B8*#MN NNN%B47)#LMN33A66 7s7AB  D;AC-,D6-+DD6DD66D;c/n0nUHn[U[5(a2[U5unnURU5 UR U5 MJ[U[ 5(aUR[ U55 M{[RS[U55 M [U5n U (a4URXUS9 [![#UR%555 gg![RaQ SUlUVs/sHn[U[5(dMUPM Os snfn nU (aURU 5 Nf=f![![#UR%555 f=f)Nz+Unknown operation type in tracing queue: %srT)rqr,r2rxrr+r1rrrr*_send_multipart_reqr;rr0rrrr) rrrrpartsopened_files_dictrpartr acc_multipartrun_opss r?_multipart_ingest_opsClient._multipart_ingest_ops1sH13:<B"455LBO  T"!((6B ;<< PQST JDQSHU9?  ?((!G)T"3":":"<=>  22 8,0((+V"z">T/U2VV..w7 8T"3":":"<=>s68C,,E D/(D/. EEEE$E8)rrc U(dU(dgU=(d [Vs/sHoPRU5PM nnU=(d [Vs/sHoPRUSS9PM nnU(aLUHEnURS5(aURS5(aM1[R"S5e AU(aLUHEnURS5(aURS5(aM1[R"S5e AU(anU(agUVs0sHoUSU_M nn/n UHGnUSU;a*UR 5HupU cM XUSU 'M M6U R U5 MI AU nU(d!URU5nURUSS 9nU(dU(dgURU5 URU5 [[[R"S U5S U5555n U Hn [U [5(dMU R(dM-U RR!5HFn[U["5(dM[US [$5(dM4U(aM=['S 5e M UR)U 5 gs snfs snfs snf)a Batch ingest/upsert multiple runs in the Langsmith system. Args: create (Optional[Sequence[Union[ls_schemas.Run, RunLikeDict]]]): A sequence of `Run` objects or equivalent dictionaries representing runs to be created / posted. update (Optional[Sequence[Union[ls_schemas.Run, RunLikeDict]]]): A sequence of `Run` objects or equivalent dictionaries representing runs that have already been created and should be updated / patched. pre_sampled (bool, default=False): Whether the runs have already been subject to sampling, and therefore should not be sampled again. Raises: LangsmithAPIError: If there is an error in the API request. !!! note The run objects MUST contain the `dotted_order` and `trace_id` fields to be accepted by the API. Example: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() _session = "__test_batch_ingest_runs" trace_id = uuid4() trace_id_2 = uuid4() run_id_2 = uuid4() current_time = datetime.datetime.now(datetime.timezone.utc).strftime( "%Y%m%dT%H%M%S%fZ" ) later_time = ( datetime.datetime.now(datetime.timezone.utc) + timedelta(seconds=1) ).strftime("%Y%m%dT%H%M%S%fZ") runs_to_create = [ { "id": str(trace_id), "session_name": _session, "name": "run 1", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id)}", "trace_id": str(trace_id), "inputs": {"input1": 1, "input2": 2}, "outputs": {"output1": 3, "output2": 4}, }, { "id": str(trace_id_2), "session_name": _session, "name": "run 3", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id_2)}", "trace_id": str(trace_id_2), "inputs": {"input1": 1, "input2": 2}, "error": "error", }, { "id": str(run_id_2), "session_name": _session, "name": "run 2", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id)}." f"{later_time}{str(run_id_2)}", "trace_id": str(trace_id), "parent_run_id": str(trace_id), "inputs": {"input1": 5, "input2": 6}, }, ] runs_to_update = [ { "id": str(run_id_2), "dotted_order": f"{current_time}{str(trace_id)}." f"{later_time}{str(run_id_2)}", "trace_id": str(trace_id), "parent_run_id": str(trace_id), "outputs": {"output1": 4, "output2": 5}, }, ] client.multipart_ingest(create=runs_to_create, update=runs_to_update) ``` NT)rrwrzNMultipart ingest requires trace_id and dotted_order to be set in create dicts.zNMultipart ingest requires trace_id and dotted_order to be set in update dicts.r<rtc3:# UHn[SU5v M g7frrrs r?r*Client.multipart_ingest..sM '44 rc3:# UHn[SU5v M g7frrrs r?rr sN#'55rrr)rkrbrr;r|rrxr|ror-rrrrqr,rrrr rur)rrrrrr`rr create_by_idstandalone_updatesrmrnrrrs r?multipart_ingestClient.multipart_ingestYsB& 2MQ 0DsJ=K!Kr)attemptsrrc ^URnURnUbUbX40nO URnUR5Hup[ SUS-5Hn [ R "U[S9n U RS::aU R5n OU n [RSUSU35 URSUS3U 0URE[U SU R0ES .SUS 9 M M g![ R"a  M[ R$[ R&[ R(4aAn X:Xa0[R+S U 35 UR-U 5 Sn A GMSn A GM$Sn A f[.an[0R2"[5U5U5nS R7U5R95n[R+S U35 O1![.a$ [R+S [;U535 Of=fUR-U5 SnA GMSnAff=f) Nrboundary-1zSending multipart request to rr/runs/multipartrrrrrz!Failed to multipart ingest runs: r)rrrrrrqtb_multipartMultipartEncoderr&rz to_stringrrrrr content_typer;rrrrrtrrrrrrrru)raccrrrrrrrrrencoderrexcrrrs r?rClient._send_multipart_req s ;;  7#63:2DI,,I.7oo.? *NQ1 --,==eiXG{{j0&002&LL77GW_V`a--)*/:$(("&--( )> .0D0D((,-!). 1./@4 665544.. ! )J3%'PQ;;C@@  V)2)H)HaRS)T#%77>#:#A#A#C)J8*'UV$V)J4PQ7)'TUV77: sJBC11H, 2H,;.E77 H,AGH'+H H' H H''H,rc @SR[US/55nURR5HupVUR S5 [ SUS-5H}n0UR EUS[3SU(a[US5OSU(a[US5OSS.En[RS U35 URS US 3UUS .SUS 9 M M g![Ra  M[R[R[R 4aAn Xs:Xa0[R#SU 35 UR%U 5 Sn A GMSn A GM Sn A f[&an [(R*"[-U 5U 5n SRU 5R/5n [R#SU 35 O1![&a$ [R#S[1U 535 Of=fUR%U 5 Sn A GMSn A ff=f)zASend a zstd-compressed multipart form data stream to the backend.rrrrzmultipart/form-data; boundary=zstdr)z X-API-KEYrzContent-EncodingzX-Pre-Compressed-SizezX-Post-Compressed-Sizez3Sending compressed multipart request with context: rrrrz,Failed to send compressed multipart ingest: N)rrVrrr5rrr&rtrrrr;rrrrrtrrrrrrru) r data_streamcompressed_traces_inforrrrrrCrrrrs r?_send_compressed_multipart_req%Client._send_compressed_multipart_reqG s '+y""EF $ 4 4 : : < G   Q Q1 -:--%,*H (T,2 6 6q 9:!# 6 6q 9:!#G LLMhZX--")?3$/'.(,-!). ?.!=F 665544.. ! J3%P;;C@@   )2)H)HaRS)T#%77>#:#A#A#CJ8*U%J4PQ7)T77: sJ A8C""H:2H,.E(( H6AG H+G< 9H;G< <HH)r1rSrQend_timerrNrOrrPtagsrrreference_example_idrrc [US5UUURSS5URSS5URSS5U U URSS5URSS5S. nUbUR5US 'U (abU R5HIunn[ U[ 5(dM[ US [ 5(dM7U (aM@[S 5e U US 'URSL=(d URSL=(a USSL=(a USSLnURU/S S9(dgUbUR5US'OH[RR[RR5R5US'UbUUS'UbURU5US'Ub1U(d[ R""U5nUR%U5US'U bU US'US(aDUR'U/5 USR)S5=n(aUR+U5USS'UbUUS'UR,(aoUR)S5(dYUR. UR0R3SU45 UR55(aUR75 SSS5 gUR9UUUS9 g!,(df  g=f)a5Update a run in the LangSmith API. Args: run_id (Union[UUID, str]): The ID of the run to update. name (Optional[str]): The name of the run. run_type (Optional[str]): The type of the run (e.g., llm, chain, tool). start_time (Optional[datetime.datetime]): The start time of the run. end_time (Optional[datetime.datetime]): The end time of the run. error (Optional[str]): The error message of the run. inputs (Optional[Dict]): The input values for the run. outputs (Optional[Dict]): The output values for the run. events (Optional[Sequence[dict]]): The events for the run. extra (Optional[Dict]): The extra information for the run. tags (Optional[List[str]]): The tags for the run. attachments (Optional[Dict[str, Attachment]]): A dictionary of attachments to add to the run. The keys are the attachment names, and the values are Attachment objects containing the data and mime type. reference_example_id (Optional[Union[str, uuid.UUID]]): ID of the example that was the source of the run inputs. Used for runs that were part of an experiment. api_key (Optional[str]): The API key to use for this specific run. api_url (Optional[str]): The API URL to use for this specific run. **kwargs (Any): Kwargs are ignored. Returns: None Examples: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() project_name = "__test_update_run" start_time = datetime.datetime.now() revision_id = uuid4() run: dict = dict( id=uuid4(), name="test_run", run_type="llm", inputs={"text": "hello world"}, project_name=project_name, api_url=os.getenv("LANGCHAIN_ENDPOINT"), start_time=start_time, extra={"extra": "extra"}, revision_id=revision_id, ) # Create the run client.create_run(**run) run["outputs"] = {"output": ["Hi"]} run["extra"]["foo"] = "bar" run["name"] = "test_run_updated" # Update the run client.update_run(run["id"], **run) ``` run_idrwN parent_run_idr session_idr) r<r1rSrwr*rr&rPr+rrQrrrTrtr%rrNrOrrPrr'rrur)r{r_ isoformatrrqrr rurr&r|r[r\r]r^rXr;rWrYrorrZrr*r)rxrr _update_run)rr)r1rSrQr%rrNrOrrPr&rrr'rrrr_r use_multipartrs r? update_runClient.update_run s^68,  :t4#ZZ>"JJ~t< **\48"JJ~t<   !!+!5!5!7D  !,!2!2!4 :z511":a=$7788$o "5#.D    t + Qt/E/ET/Q 1Z , 1^$D0  (($t(<   '113D '0044X5F5F5J5JKUUWD   !DM  !226:DN   "//8"44W=DO  #DN =  $ $dV ,=,,Z88x8,0,C,CH,MW j)  ++?D' (  ) )&**=V2W2W**$$++WdO<4466..0 +*   T7G  D+*s AK K-c URSL=(d URSL=(a USSL=(a USSLnURbURRU5 gU(Ga[ SUS9nURbUcUc[ U5unn[ RSSUR5 URR URc [S5e[UUR[5nU(a9UR=RS- slURR5 SSS5 [![#UR%555 gURb[ RSS UR&UR(5 UR*bQURR-[/USUUUUR1UR2R555S 95 gURR-[/USUUUS 95 ggUR7XUS 9 g!,(df  GN=f) Nrwrru)rpayloadrrrrrrr)rr&r%r0r0r2rrrrr'rur.r&rrZrrrrwr<r+rr r-r,r_update_run_non_batch) r run_updaterrr/rrrrs r?r-Client._update_run s<  t + Qt/E/ET/Q 7:&d2 7>*$6     (    ( ( 4 .*UM&&2OO L!"  K"** ++00119(^ D&..! H  ..::a?:226681T,"5"5"789##/ E!**!$$  %%1&&**(&~6)$+$+)-)B)B $ 0 0 A A C*  &&**(&~6)$+$+ )0:  & &zG & TW10s A4I IcUcUbfU=(d URnU=(d URn0URE[U0EnUR SUSUS3[ U5US.S9 gUR R5H@upx0URE[U0EnUR SUSUS3[ U5US.S9 MB g)NPATCH/runs/r<rr$)rrrrrr'rr) rr5rrrrrCrrs r?r4Client._update_run_non_batchn s  '"5$4 N$4 N-->G  % %!"&D)9(:;' 3&  & 150D0D0J0J0L, mm} ))$oVJt,<+=> +J 7#*$* 1Mr>cURcgURc [S5eSSKJnJn U"USSS9unnUb:SnUR URUUUS9nURRU5 UR(aI[UR5n[R"U5upURRU5 gg![a URXEUS9 Nwf=f)z3Force flush the currently buffered compressed runs.Nz>Run compression is enabled but request pool futures is not setr)r'_tracing_thread_drain_compressed_bufferr)rrr)r&r(rurrr<rr#rzrrcfwaitdifference_update) rrrr<final_data_streamr"rfuturesdoner.s r?flush_compressed_tracesClient.flush_compressed_traces s  ! ! )  == P    4 Q   "  (F 5<<77%*% =  !!&) ==4==)Gggg&GD MM + +D 1   33%4 s7CC/.C/c@UR(a6UR UR(aUR5 SSS5 URbUR 5 gUR bUR R5 gg!,(df  NU=f)z;Flush either queue or compressed buffer, depending on mode.N)rr*r)rr&rCrrrs r?flush Client.flush sv  ) )**''..0+  ! ! -  ( ( *    +    # # %, +*s "B BcURSURURS9n[R"[ 5n0n[ UR5n[USS9HnURc$[R"SURS35eURRS5Vs1sH nSU;dM URSS 5S iM" nnXX;dMURUR:wdMX6RRU5 XdUR'M URUR/5UlUR#5HupXU lM U$s snf) zLoad child runs for a given run. Args: run (Run): The run to load child runs for. Returns: Run: The run with loaded child runs. Raises: LangSmithError: If a child run has no parent. F)is_rootr+rwcUR$r)r)rs r?)Client._load_child_runs.. s!..r>)rz Child run z has no parentrZr) list_runsr+rwrrrrtr<sortedr*r;rrryrxr_ child_runsr) rr`rQtreemapr run_id_str child_runseg ancestor_idsr)childrens r?_load_child_runsClient._load_child_runs sU^^cnns||$   # #D ) 13[  ( I&&.-- 9<<..WXX %1177<) project_idr~rSrwr'queryfilter trace_filter tree_filterrIr*rQrrun_idsselectrc+# /n[U[R[45(aUR U5 O&[U[ 5(aUR U5 UbQ[U[5(aU/nUR UVs/sHnURUS9RPM sn5 /SQnU=(d UnSU;a[R"S[5 U(aUOSUU(aU/OSUUUU U U U (aU R5OSU UUUUS.UEnUR5VVs0sHunnUcM UU_M nnn[URSUS95HSunn[!US UR"S 9n[$R&"SS U0UDS UR(0D6v UcMHUS -U:dMS g gs snfs snnf7f)aqList runs from the LangSmith API. Args: project_id (Optional[Union[UUID, str], Sequence[Union[UUID, str]]]): The ID(s) of the project to filter by. project_name (Optional[Union[str, Sequence[str]]]): The name(s) of the project to filter by. run_type (Optional[str]): The type of the runs to filter by. trace_id (Optional[Union[UUID, str]]): The ID of the trace to filter by. reference_example_id (Optional[Union[UUID, str]]): The ID of the reference example to filter by. query (Optional[str]): The query string to filter by. filter (Optional[str]): The filter string to filter by. trace_filter (Optional[str]): Filter to apply to the ROOT run in the trace tree. This is meant to be used in conjunction with the regular `filter` parameter to let you filter runs by attributes of the root run within a trace. tree_filter (Optional[str]): Filter to apply to OTHER runs in the trace tree, including sibling and child runs. This is meant to be used in conjunction with the regular `filter` parameter to let you filter runs by attributes of any run within a trace. is_root (Optional[bool]): Whether to filter by root runs. parent_run_id (Optional[Union[UUID, str]]): The ID of the parent run to filter by. start_time (Optional[datetime.datetime]): The start time to filter by. error (Optional[bool]): Whether to filter by error status. run_ids (Optional[Sequence[Union[UUID, str]]]): The IDs of the runs to filter by. select (Optional[Sequence[str]]): The fields to select. limit (Optional[int]): The maximum number of runs to return. **kwargs (Any): Additional keyword arguments. Yields: The runs. Examples: ```python # List all runs in a project project_runs = client.list_runs(project_name="") # List LLM and Chat runs in the last 24 hours todays_llm_runs = client.list_runs( project_name="", start_time=datetime.now() - timedelta(days=1), run_type="llm", ) # List root traces in a project root_runs = client.list_runs(project_name="", is_root=1) # List runs without errors correct_runs = client.list_runs(project_name="", error=False) # List runs and only return their inputs/outputs (to speed up the query) input_output_runs = client.list_runs( project_name="", select=["inputs", "outputs"] ) # List runs by run ID run_ids = [ "a36092d2-4ad5-4fb4-9c0d-0dba9a2ed836", "9398e6be-964f-4aa4-8ae9-ad78cd4b7074", ] selected_runs = client.list_runs(id=run_ids) # List all "chain" type runs that took more than 10 seconds and had # `total_tokens` greater than 5000 chain_runs = client.list_runs( project_name="", filter='and(eq(run_type, "chain"), gt(latency, 10), gt(total_tokens, 5000))', ) # List all runs called "extractor" whose root of the trace was assigned feedback "user_score" score of 1 good_extractor_runs = client.list_runs( project_name="", filter='eq(name, "extractor")', trace_filter='and(eq(feedback_key, "user_score"), eq(feedback_score, 1))', ) # List all runs that started after a specific timestamp and either have "error" not equal to null or a "Correctness" feedback score equal to 0 complex_runs = client.list_runs( project_name="", filter='and(gt(start_time, "2023-07-15T12:34:56Z"), or(neq(error, null), and(eq(feedback_key, "Correctness"), eq(feedback_score, 0.0))))', ) # List all runs where `tags` include "experimental" or "beta" and `latency` is greater than 2 seconds tagged_runs = client.list_runs( project_name="", filter='and(or(has(tags, "experimental"), has(tags, "beta")), gt(latency, 2))', ) ``` Nr~)app_pathcompletion_costcompletion_tokensrr%rrrPfeedback_statsfirst_token_timer<rNr1rOr*parent_run_ids prompt_cost prompt_tokensr'rSr+rQstatusr& total_cost total_tokensrw child_run_idszOThe child_run_ids field is deprecated and will be removed in following versions)rrSreference_examplerdrerfrgrI parent_runrQrr<rCrir /runs/queryr r[r\rrrr=)rqrrrsrtrxrextend read_projectr<rrDeprecationWarningr,r enumerater)r^rrbr_r)rrcr~rSrwr'rdrerfrgrIr*rQrrhrirr project_idsr1default_select body_queryrmrnir`rs r?rOClient.list_runs# s^ j499c"2 3 3   z *  D ) )   z *  #,,, ,~   EQR\T"""588\R  <)> f $ MMa"  '2{t *>%&D(&'4>*..0D#& $%& ((2'7'7'9K'9tq!Qdad'9 K  + +M + K FAsJY K..'+.:>..  QUe^ {SxLs2B G  #G/BG 7 GG A G 0G ;G )r<rCryrS project_namesrreference_example_idsrQr%rrdrerfrgrIdata_source_typecPSSKJnJn U=(d /nU(arU"5nUVs/sHnURURUS9PM nnU"U5H,nUR UR 5R5 M. SSS5 UUUUUUUU U U U U UUUS.nUR5VVs0sHunnUcM UU_M nnnURSSS[U50S 9n[R"U5 UR5$s snf!,(df  N=fs snnf) a Get aggregate statistics over queried runs. Takes in similar query parameters to `list_runs` and returns statistics based on the runs that match the query. Args: id (Optional[List[Union[UUID, str]]]): List of run IDs to filter by. trace (Optional[Union[UUID, str]]): Trace ID to filter by. parent_run (Optional[Union[UUID, str]]): Parent run ID to filter by. run_type (Optional[str]): Run type to filter by. project_names (Optional[List[str]]): List of project names to filter by. project_ids (Optional[List[Union[UUID, str]]]): List of project IDs to filter by. reference_example_ids (Optional[List[Union[UUID, str]]]): List of reference example IDs to filter by. start_time (Optional[str]): Start time to filter by. end_time (Optional[str]): End time to filter by. error (Optional[bool]): Filter by error status. query (Optional[str]): Query string to filter by. filter (Optional[str]): Filter string to apply. trace_filter (Optional[str]): Trace filter string to apply. tree_filter (Optional[str]): Tree filter string to apply. is_root (Optional[bool]): Filter by root run status. data_source_type (Optional[str]): Data source type to filter by. Returns: Dict[str, Any]: A dictionary containing the run statistics. rThreadPoolExecutor as_completedrkN)r<rCryrSrrxrQr%rrdrerfrgrIrrz /runs/statsrr$)concurrent.futuresrrrr}rxrJr<rrr'r;rr)rr<rCryrSrrrrQr%rrdrerfrgrIrrrexecutorr1rArr3rmrnrs r? get_run_statsClient.get_run_stats s;\ H!'R #%!. -OOD$5$5DOI -+73F&&v}}'9'9:4 &$ "!6$ (& 0 &%,MMOEODAqq41a4OE,,   G,-  ++H5}}G&%4Fs(D$D  :D2 D"D" D D)r~rcc[USS5=n(aO[USS5=n(aURUS9RnOQUbUnOKUbURUS9RnO.[R"5nURUS9Rn[ US5nUR SUR5SUSURS3$) akGet the URL for a run. Not recommended for use within your agent runtime. More for use interacting with runs after the fact for data analysis or ETL workloads. Args: run (RunBase): The run. project_name (Optional[str]): The name of the project. project_id (Optional[Union[UUID, str]]): The ID of the project. Returns: str: The URL for the run. r+Nrrkz/o/z /projects/p/z/r/z ?poll=true)rVr}r<r;rr{r_get_tenant_id)rr`r~rcr+r session_id_s r? get_run_urlClient.get_run_url= s*!lD9 9: 9 $S.$? ?\ ?** *EHHJ  ##J  %** *EHHJ#668L** *EHHJz<8 ~~c$"5"5"7!8 [MR  $ r>)share_idc6[US5n[U5U=(d [[R"55S.nUR SSUS3UR US9n[ R"U5 UR5SnURSUS 3$) aGet a share link for a run. Args: run_id (Union[UUID, str]): The ID of the run to share. share_id (Optional[Union[UUID, str]]): Custom share ID. If not provided, a random UUID will be generated. Returns: str: The URL of the shared run. r))r) share_tokenPUTr9/sharerCrr/public//r) r{rtrrrDrrr;rrr)rr)rrun_id_rrrs r? share_runClient.share_runc s68,'l#8s4::<'8 ,, WIV $MM -  ++H5mmom4 ..!+b99r>cURSS[US5S3URS9n[R"U5 g)ztDelete share link for a run. Args: run_id (Union[UUID, str]): The ID of the run to unshare. Returns: None DELETEr9r)rrCNrr{rr;r)rr)rs r? unshare_runClient.unshare_run} sG,, Xfh/0 7MM-  ++H5r>cURSS[US5S3URS9n[R"U5 UR 5nUbSU;agUR SUSS 3$) zRetrieve the shared link for a specific run. Args: run_id (Union[UUID, str]): The ID of the run. Returns: Optional[str]: The shared link for the run, or None if the link is not available. rr9r)rrNrrr)rr{rr;rrr)rr)rrJs r?read_run_shared_linkClient.read_run_shared_link s,, Xfh/0 7MM-  ++H5 >]&8..!&*?)@CCr>c@UR[US55nUSL$)zGet share state for a run. Args: run_id (Union[UUID, str]): The ID of the run. Returns: bool: True if the run is shared, False otherwise. r)N)rr{)rr)rs r? run_is_sharedClient.run_is_shared s&((&()CD4r>c[USSS9up4SUS3nUbUS[US53- nURSUURS 9n[R "U5 [ R"S 0UR5DS UR0D6$) aGet shared runs. Args: share_token (Union[UUID, str]): The share token or URL of the shared run. run_id (Optional[Union[UUID, str]]): The ID of the specific run to retrieve. If not provided, the full shared run will be returned. Returns: Run: The shared run. rr`rr/runrnr)rrrr=) rr{rrr;rrbr_rr)rrr)r.rrxrs r?read_shared_runClient.read_shared_run s,K%H *T*   a234 4D,,  MM-  ++H5~~J J4>>JJr>c## U(aSU0O0n[USSS9upEURSUS3US9H)n[R"S 0UDSUR0D6v M+ g 7f) zGet shared runs. Args: share_token (Union[UUID, str]): The share token or URL of the shared run. run_ids (Optional[List[str]]): A list of run IDs to filter the results by. Yields: A shared run. r<rr`rrrzr{rNr=)rr)rbr_r)rrrhr r.rr`s r?list_shared_runsClient.list_shared_runs si#*gr+K%H 22zl+ .T3 C..A3A$..A A sAA dataset_namec tUcUc [S5eUcURUS9RnURSS[ US5S3UR S9n[ R"U5 UR5n[[R0UESURS [ US S 5S 30E5$) aRetrieve the shared schema of a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. Either `dataset_id` or `dataset_name` must be given. dataset_name (Optional[str]): The name of the dataset. Either `dataset_id` or `dataset_name` must be given. Returns: ls_schemas.DatasetShareSchema: The shared schema of the dataset. Raises: ValueError: If neither `dataset_id` nor `dataset_name` is given. /Either dataset_id or dataset_name must be givenrr /datasets/ dataset_idrrrrrzresponse.share_token/d) ru read_datasetr<rr{rr;rrrrbDatasetShareSchemar)rrrrds r?read_dataset_shared_schema!Client.read_dataset_shared_schema s(  ,"6NO O  ** *EHHJ,, *l;c |UcUc [S5eUcURUS9RnS[U50nUR SS[ US5S3UR US9n[R"U5 UR5n[[R0UESURS US S 30E5$) aGet a share link for a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. Either `dataset_id` or `dataset_name` must be given. dataset_name (Optional[str]): The name of the dataset. Either `dataset_id` or `dataset_name` must be given. Returns: ls_schemas.DatasetShareSchema: The shared schema of the dataset. Raises: ValueError: If neither `dataset_id` nor `dataset_name` is given. rrrrrrrrrrr)rurr<rtrr{rr;rrrrbrr)rrrrrrs r? share_datasetClient.share_dataset s(  ,"6NO O  ** *EHHJ #j/ ,, *l;cURSS[US5S3URS9n[R"U5 g)zDelete share link for a dataset. Args: dataset_id (Union[UUID, str]): The ID of the dataset to unshare. Returns: None rrrrrNr)rrrs r?unshare_datasetClient.unshare_dataset* sG,, *l;c[XR5up#URSSUS3URS9n[R "U5 [ R"S0UR5DURSUS3S.D6$)zGet shared datasets. Args: share_token (Union[UUID, str]): The share token or URL of the shared dataset. Returns: Dataset: The shared dataset. rr /datasetsrr)r _public_pathr=) rrrrr;rrbrFrr)rrr.rrs r?read_shared_datasetClient.read_shared_dataset: s,KF ,, zl) ,MM-  ++H5!! mmo nn#K=3  r>) example_idsrc #*# 0nUbUVs/sHn[U5PM snUS'[URS[US5S3US95H;upg[R "S 0UDSUR 0D6v UcM0US-U:dM; g gs snf7f) abGet shared examples. Args: share_token (Union[UUID, str]): The share token or URL of the shared dataset. example_ids (Optional[List[UUID, str]], optional): The IDs of the examples to filter by. limit (Optional[int]): Maximum number of examples to return, by default None. Returns: List[ls_schemas.Example]: The list of shared examples. Nr<rr /examplesrrrr=)rtrrr{rbExampler)rrrrrr<rexamples r?list_shared_examplesClient.list_shared_examplesS s"  ".9:kCGk:F4L#  $ $8K?@ J %  JA $$IwI$..I I QUe^ ;s BBAB>B B)rr1 name_containsrc## X#US.n[US5n[URSUS3US95H;up[R"S 0U DSUR 0D6v UcM0US-U:dM; g g7f) aList shared projects. Args: dataset_share_token (str): The share token of the dataset. project_ids (Optional[List[Union[UUID, str]]]): List of project IDs to filter the results, by default None. name (Optional[str]): Name of the project to filter the results, by default None. name_contains (Optional[str]): Substring to search for in project names, by default None. limit (Optional[int]): Maximum number of projects to return, by default None. Yields: The shared projects. )r<r1rdataset_share_tokenrz/datasets/sessionsrrNrr=)r{rrrbTracerSessionResultr) rrrr1rrrrrprojects r?list_shared_projectsClient.list_shared_projectsq s*$MR24IJ #  $ $;-'9: %  JA 00U7UdnnU U QUe^ sAA1!A1,A1)r+rupsert project_extrareference_dataset_idcURS3nUnU(a0U=(d 0ESU0EnUUU[[R"55S.n 0n U(aSU S'UbXiS'UR SU0UR ESS 0E[ U 5S 9n [R"U 5 [R"S 0U R5DS UR0D6$) a3Create a project on the LangSmith API. Args: project_name (str): The name of the project. project_extra (Optional[dict]): Additional project information. metadata (Optional[dict]): Additional metadata to associate with the project. description (Optional[str]): The description of the project. upsert (bool, default=False): Whether to update the project if it already exists. reference_dataset_id (Optional[Union[UUID, str]): The ID of the reference dataset to associate with the project. Returns: TracerSession: The created project. /sessionsr)r1rPr+r<TrrrrrrCrrr=) rrtrrrDrrr'r;rrb TracerSessionrr) rr~r+rrrrendpointrPr rrs r?create_projectClient.create_project s.ll^9- ; ;j(;E &djjl#    #F8   ++?' (,,  It}}In6HIT" -  ++H5''T(--/TT^^TTr>)r1r+rrr%cURS[US53nUnU(a0U=(d 0ESU0EnUUUU(aUR5OSS.n URSU0URESS0E[ U 5S 9n [ R"U 5 [R"S 0U R5DS UR0D6$) aUpdate a LangSmith project. Args: project_id (Union[UUID, str]): The ID of the project to update. name (Optional[str]): The new name to give the project. This is only valid if the project has been assigned an end_time, meaning it has been completed/closed. description (Optional[str]): The new description to give the project. metadata (Optional[dict]): Additional metadata to associate with the project. project_extra (Optional[dict]): Additional project information. end_time (Optional[datetime.datetime]): The time the project was completed. Returns: TracerSession: The updated project. /sessions/rcrN)r1rPr+r%r8rrrrr=) rr{r,rrr'r;rrbrrr) rrcr1r+rrr%rrPr rs r?update_projectClient.update_project s<ll^:hz<.P-QR ; ;j(;E&08**,d   ,,  It}}In6HIT" -  ++H5''T(--/TT^^TTr>cURb UR$URSSSS0S9nUR5n[U[5(aS[ U5S:aD[ R"S 0USDSUR0D6nURUlUR$g![a(n[RS[U5S S 9 SnAgSnAff=f) Nrrrrrrrz*Failed to get tenant ID from LangSmith: %sTrr=)rrrrqrrzrbrr tenant_idrrrru)rrrJtracer_sessionrs r?rGClient._get_optional_tenant_id s ?? &?? " 00{GQ<1H]]_F&$''CK!O!+!?!?"Qi"+/>>"#1":":&    LL? ?r>)rcr~)rcr~ include_statscSnSS0nUbUS[US53- nOUbX%S'O [S5eX5S'URS XES 9nUR5n[ U[ 5(aP[ U5S :Xa[R"S US 35e[R"S0US DSUR0D6$[R"S0UR5DSUR0D6$)aRead a project from the LangSmith API. Args: project_id (Optional[str]): The ID of the project to read. project_name (Optional[str]): The name of the project to read. Only one of project_id or project_name may be given. include_stats (bool, default=False): Whether to include a project's aggregate statistics in the response. Returns: TracerSessionResult: The project. rrrrnrcr1'Must provide project_name or project_idrrrrzProject not foundrr=) r{rurrrqrrzr;rrbrr)rrcr~rrxrrrJs r?r}Client.read_project s*")1  ! a\:;< rccXURUS9 g![Ra gf=f)zCheck if a project exists. Args: project_name (str): The name of the project to check for. project_id (Optional[str]): The ID of the project to check for. Returns: bool: Whether the project exists. rkFT)r}r;r)rr~rcs r? has_projectClient.has_project6s7    <  8..   ))c P^[R"S[SS9 SSKJnJn SSKnTRUUS/SQS 9n/n/nU4S jn S n Sn U"5n /n UGHEnS UR0URR5VVs0sH unnS U3U_M snnEUR=(d 0R5VVs0sH unnSU3U_M snnEUR(a'URUR- R5OSURUR S.EnUR"(aUR%UR"R5VVs0sH9unnUS:XaUR'S5(aM$SU3UR'S5_M; snn5 UR"R'S5(a*UR"SR'S5=n(aUUS'UR(aUR)UR5 O#[*R-SUR S35 [/U5U -S:Xa2XX-=n(a%U R)U R1U U55 X- n UR)U5 GMH XS(a#U R)U R1XU S55 SSS5 UR2"U5R5S 5nU"W 5VVs/sHnUR75HnUPM M nnnU(a2UR2"U5R5S 5nUR9USSS9nUR:"UR=SS95$s snnfs snnfs snnf!,(df  N=fs snnf)a,Read the record-level information from an experiment into a Pandas DF. !!! note This will fetch whatever data exists in the DB. Results are not immediately available in the DB upon evaluation run completion. Feedback score values will be returned as an average across all runs for the experiment. Non-numeric feedback scores will be omitted. Args: project_id (Optional[Union[UUID, str]]): The ID of the project. project_name (Optional[str]): The name of the project. Returns: pd.DataFrame: A dataframe containing the test results. z%Function get_test_results is in beta.r) stacklevelrrNT)r<r'rNrOrrorQr%)rcr~rIric >TRUS9nUVVVs/sHMnSUR0UR=(d 0R5VVs0sH up4SU3U_M snnEPMO snnn$s snnfs snnnf)N)r example_idz reference.) list_examplesr<rOr)batchexamplesrrmrnrs r?fetch_examples/Client.get_test_results..fetch_exampleszs))e)>3CD3C41as|Q3CD 78iio25L5L5NO5NTQ!~q(5NO  ::all2AAC!WW$$ ##JJ)*(8(8(>(>(@(@1$%KAEE*4E:isOQUU5\9(@''++F33$%$4$4V$<$@$@$LLL'/G ))&&q'='=>NNT!$$/L#MN{#j0A5 +V5H IIuIx~u'MN,s#EJ7#x~67?STUQ"RLL)33LA !-g!6 !6vV]]_6F_F!6   o6@@NJ"((tQU(VI  !2!2)!2!DEE[EO#" !T sD3NM? *NN BN$#N  N $D(N$N"?N Nc ## SU b [U S5OS0n UbXS'UbX+S'UbX;S'UbUb [S5eXKS'O UbURUS 9RnXKS'UbXkS 'UbX{S 'UbXS 'U b[R "U 5U S '[ URSU S95H;up[R"S0U DSUR0D6v U cM0U S-U :dM; g g7f)aList projects from the LangSmith API. Args: project_ids (Optional[List[Union[UUID, str]]]): A list of project IDs to filter by, by default None name (Optional[str]): The name of the project to filter by, by default None name_contains (Optional[str]): A string to search for in the project name, by default None reference_dataset_id (Optional[List[Union[UUID, str]]]): A dataset ID to filter by, by default None reference_dataset_name (Optional[str]): The name of the reference dataset to filter by, by default None reference_free (Optional[bool]): Whether to filter for only projects not associated with a dataset. limit (Optional[int]): The maximum number of projects to return, by default None metadata (Optional[Dict[str, Any]]): Metadata to filter by. Yields: The projects. Raises: ValueError: If both reference_dataset_id and reference_dataset_name are given. rNrUr<r1rzGOnly one of reference_dataset_id or reference_dataset_name may be givenreference_datasetrreference_freerdataset_versionrrrrrr=) minrurr<rrrrrbrr)rrr1rrreference_dataset_namerrrrrrrrs r? list_projectsClient.list_projectss@P (9S_s"   "&4L  !6N  $&3? #  +%1 ;+?& ' # /#'#4#43$5$b !+?& '  %'5# $  $&3? #  &(7$ %  !%H!5F: #  $ $[ $ @ JA00U7UdnnU U QUe^  sCC+C+&C+cUb#[URUS9R5nOUc [S5eUR SS[ US53UR S9n[R"U5 g)aDelete a project from LangSmith. Args: project_name (Optional[str]): The name of the project to delete. project_id (Optional[str]): The ID of the project to delete. Returns: None Raises: ValueError: If neither project_name or project_id is provided. Nrkrrrrcr) rtr}r<rurr{rr;r)rr~rcrs r?delete_projectClient.delete_projects{$  #T..L.ILLMJ  FG G,, *l;< =MM-  ++H5r>)r+r, inputs_schemaoutputs_schematransformationsrc.S[R"50U=(d 0EnUURUSS[R"50U=(d 0E0S.nUbX(S'UbXHS'UbXXS'URSS0URES S 0E[ R "U5S 9n [R"U 5 U R5n U RS5=(d UU S'[R"S 0U DURUR5S .D6$)a(Create a dataset in the LangSmith API. Args: dataset_name (str): The name of the dataset. description (Optional[str]): The description of the dataset. data_type (DataType, default=DataType.kv): The data type of the dataset. inputs_schema (Optional[Dict[str, Any]]): The schema definition for the inputs of the dataset. outputs_schema (Optional[Dict[str, Any]]): The schema definition for the outputs of the dataset. transformations (Optional[List[DatasetTransformation]]): A list of transformations to apply to the dataset. metadata (Optional[dict]): Additional metadata to associate with the dataset. Returns: Dataset: The created dataset. Raises: requests.HTTPError: If the request to create the dataset fails. rer)r1r,r(rPr+inputs_schema_definitionoutputs_schema_definitionrrrrrrBr=)rfrgrrrrrr;rrrrbrFrrG) rrr+r,r&r'r(rdatasetr json_responses r?create_datasetClient.create_dataset"s.Fv==?THNPRT ".v==?~2 #   "%0M "  $2?. /  %3A/ 0,,  It}}In6HIw' -  ++H5  $1$5$5j$A$MX j!!!  nn335  r>rrcXURXS9 g![Ra gf=f)aCheck whether a dataset exists in your tenant. Args: dataset_name (Optional[str]): The name of the dataset to check. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to check. Returns: bool: Whether the dataset exists. r0TF)rr;r)rrrs r? has_datasetClient.has_datasetjs3"    <  O..  rcSnSS0nUbUS[US53- nOUbXS'O [S5eURSUUS 9nUR5n[ U[ 5(a_[ U5S :Xa[R"S US 35e[R"S0US DURUR5S .D6$[R"S0UDURUR5S .D6$)zRead a dataset from the LangSmith API. Args: dataset_name (Optional[str]): The name of the dataset to read. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to read. Returns: Dataset: The dataset. rrrrnrr1'Must provide dataset_name or dataset_idrrrrArrBr=) r{rurrrqrrzr;rrbrFrrG)rrrrxrrrJs r?rClient.read_datasets$")1  ! a\:;< c Uc'Uc [S5eURUS9Rn[US5nUR SSUS3UR [ U[R5(aUR5OU[ U[R5(aUR5OUS.S9n[R"U5 [R"S 0UR5D6$) a=Get the difference between two versions of a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. from_version (Union[str, datetime.datetime]): The starting version for the diff. to_version (Union[str, datetime.datetime]): The ending version for the diff. Returns: DatasetDiffInfo: The difference between the two versions of the dataset. Examples: ```python # Get the difference between two tagged versions of a dataset from_version = "prod" to_version = "dev" diff = client.diff_dataset_versions( dataset_name="my-dataset", from_version=from_version, to_version=to_version, ) # Get the difference between two timestamped versions of a dataset from_version = datetime.datetime(2024, 1, 1) to_version = datetime.datetime(2024, 2, 1) diff = client.diff_dataset_versions( dataset_name="my-dataset", from_version=from_version, to_version=to_version, ) ``` &Must provide either dataset name or IDrrrrz/versions/diff) from_version to_versionrCrr=)rurr<r{rrrqr[r,r;rrbDatasetDiffInfor)rrrr9r:dsidrs r?diff_dataset_versionsClient.diff_dataset_versionssX  # !IJJ** *EHHJ L1,, n -MM",0A0ABB!**,%"*h.?.?@@((*#  - " ++H5))c JSnUbO(UbURUS9RnO [S5eURSUS[ US5S35nUR R 5RS5Vs/sHn[R"U5PM nnU$s snf) aDownload a dataset in OpenAI Jsonl format and load it as a list of dicts. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to download. dataset_name (Optional[str]): The name of the dataset to download. Returns: list[dict]: The dataset loaded as a list of dicts. Raises: ValueError: If neither dataset_id nor dataset_name is provided. rrr5rrnrz /openai_ftr) rr<rurr{textrryrr)rrrrxrliner,s r?read_dataset_openai_finetuning%Client.read_dataset_openai_finetunings(  !   %** *EHHJFG G,, fAhz<89 D 19 0C0C0E0K0KD0QR0Q4::d#0QRSs< B ) dataset_idsr,rdataset_name_containsrrc#p# SUb [US5OS0nUbXS'UbX'S'UbX7S'UbXGS'Ub[R"U5US'[UR S US 95HJup[ R "S 0U DURUR5S .D6v UcM?US -U:dMJ g g7f)aList the datasets on the LangSmith API. Args: dataset_ids (Optional[List[Union[UUID, str]]]): A list of dataset IDs to filter the results by. data_type (Optional[str]): The data type of the datasets to filter the results by. dataset_name (Optional[str]): The name of the dataset to filter the results by. dataset_name_contains (Optional[str]): A substring to search for in the dataset names. metadata (Optional[Dict[str, Any]]): A dictionary of metadata to filter the results by. limit (Optional[int]): The maximum number of datasets to return. Yields: The datasets. rNrUr<r,r1rrrrrBrr=) rrrrrrbrFrrG) rrEr,rrFrrrrr,s r? list_datasetsClient.list_datasetss< (9S_s"   "&4L  "+;   #)6N ,&;? #  !%H!5F: #  $ $[ $ @ JA$$..779   QUe^ sB B6&B61B6)rrcUbURUS9RnUc [S5eURSS[ US53UR S9n[ R"U5 g)zDelete a dataset from the LangSmith API. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to delete. dataset_name (Optional[str]): The name of the dataset to delete. Returns: None Nrr8rrrr)rr<rurr{rr;r)rrrrs r?delete_datasetClient.delete_datasetMsv$  #** *EHHJ  EF F,, *l;< =MM-  ++H5r>cUbURUS9RnUc [S5eURSS[ US5S3UR UR 5US.S 9n[R"U5 g) ayUpdate the tags of a dataset. If the tag is already assigned to a different version of this dataset, the tag will be moved to the new version. The as_of parameter is used to determine which version of the dataset to apply the new tags to. It must be an exact version of the dataset to succeed. You can use the read_dataset_version method to find the exact version to apply the tags to. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to update. dataset_name (Optional[str]): The name of the dataset to update. as_of (datetime.datetime): The timestamp of the dataset to apply the new tags to. tag (str): The new tag to apply to the dataset. Returns: None Examples: ```python dataset_name = "my-dataset" # Get the version of a dataset <= a given timestamp dataset_version = client.read_dataset_version( dataset_name=dataset_name, as_of=datetime.datetime(2024, 1, 1) ) # Assign that version a new tag client.update_dataset_tags( dataset_name="my-dataset", as_of=dataset_version.as_of, tag="prod", ) ``` Nrr8rrrz/tagsas_oftagr) rr<rurr{rr,r;rrrrrOrPrs r?update_dataset_tagClient.update_dataset_tagjsZ  #** *EHHJ  EF F,, *l;)rrsearchrc ## UcURUS9RnUUb [US5OSS.n[UR S[ US5S3US95H-upg[ R"S 0UD6v UcM"US -U:dM- g g7f) a3List dataset versions. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. search (Optional[str]): The search query. limit (Optional[int]): The maximum number of versions to return. Yields: The dataset versions. NrrU)rTrrrz /versionsrrr=)rr<rrrr{rbDatasetVersion)rrrrTrrrrs r?list_dataset_versionsClient.list_dataset_versionss&  ** *EHHJ(-(9S_s $  $ $Xj,?@ J %  JA ++6g6 6 QUe^ sA6B <B B )rrrOrPcUcURUS9RnU(aU(dUcUc [S5eURSS[ US5S3X4S.S9n[ R "S 0UR5D6$) aGet dataset version by `as_of` or exact tag. Ues this to resolve the nearest version to a given timestamp or for a given tag. Args: dataset_id (Optional[ID_TYPE]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. as_of (Optional[datetime.datetime]): The timestamp of the dataset to retrieve. tag (Optional[str]): The tag of the dataset to retrieve. Returns: DatasetVersion: The dataset version. Examples: ```python # Get the latest version of a dataset client.read_dataset_version(dataset_name="my-dataset", tag="latest") # Get the version of a dataset <= a given timestamp client.read_dataset_version( dataset_name="my-dataset", as_of=datetime.datetime(2024, 1, 1), ) # Get the version of a dataset with a specific tag client.read_dataset_version(dataset_name="my-dataset", tag="prod") ``` rz/Exactly one of as_of and tag must be specified.rrrz/versionrNrr=)rr<rurr{rbrVrrQs r?read_dataset_versionClient.read_dataset_versionsL  ** *EHHJ cu}NO O,, *l;)source_api_urlrc \U=(d URn[X5up$[USS9nURU5nU=(d URnUR US9n[ RSUS35 U$![Ra Of=f[URU55nURUURUR=(d [R R"UR$UR&UR(S9nUR+UV s/sHoR,PM Os sn fsn UV s/sHoR.PM Os sn fsn UR0S9 O-![2a n [ R5SUS 35 U eS n A ff=fAU$!Af=f) a8Clone a public dataset to your own langsmith tenant. This operation is idempotent. If you already have a dataset with the given name, this function will do nothing. Args: token_or_url (str): The token of the public dataset to clone. source_api_url (Optional[str]): The URL of the langsmith server where the data is hosted. Defaults to the API URL of your current client. dataset_name (Optional[str]): The name of the dataset to create in your tenant. Defaults to the name of the public dataset. Returns: Dataset: The cloned dataset. placeholderrrrAz) already exists in your tenant. Skipping.)rr+r,r&r'r()rNrOrz)An error occurred while creating dataset z . You should delete it manually.N)rrr rr1rrr=r;rrrr.r+r,rbDataTypekvr&r'r(create_examplesrNrOr<rr) r token_or_urlr\rr source_clientdsrr,rs r?clone_public_datasetClient.clone_public_datasets,(74<<%8%V"#!   . .z :#.rww  "" "=B KK<.(QR I..    M>>zJKH)))NN,,@**=*=*@*@ ..!00 " 2 2 *G $$.67hHHh70891YY9&zz% !   ? ~N55  "s[)A>>BBBF(E7*E? E7 E! E76F(7 F!FF!!F((F+c8URUS9nUR$)N)r)rr,)rrr,s r?rqClient._get_data_typeDs!##z#:   r>c0URSU0SU0UUUS9$)aAdd an example (row) to an LLM-type dataset. Args: prompt (str): The input prompt for the example. generation (Optional[str]): The output generation for the example. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. created_at (Optional[datetime.datetime]): The creation timestamp of the example. Returns: Example: The created example inputrrNrOrr created_at)create_example)rr generationrrrls r?create_llm_exampleClient.create_llm_exampleHs54""V$z*!%! #  r>c /nUH|n[R"U5(a?UR[R"[ [ R U555 M]UR[ [U55 M~ SnUbZ[R"U5(a/[R"[ [ R U55nO[ [U5nURSU0UbSU0OSUUUS9$)a+Add an example (row) to a Chat-type dataset. Args: messages (List[Union[Mapping[str, Any], BaseMessageLike]]): The input messages for the example. generations (Optional[Union[Mapping[str, Any], BaseMessageLike]]): The output messages for the example. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. created_at (Optional[datetime.datetime]): The creation timestamp of the example. Returns: Example: The created example Nrjrrk) r;is_base_message_likerxconvert_langchain_messagerrbBaseMessageLikerrm) rmessages generationsrrrl final_inputmessagefinal_generationss r?create_chat_exampleClient.create_chat_examplejs8 G,,W55""66Z77A ""4g#67 !  ",,[99$,$F$F33[A%!%){$;!""[)1B1N,-TX!%!#  r>c<UcURUS9RnSnURU5nU[RR :XaUR S:wa[SUR S35e[R"UR5nSU0nUR(dSnGO[R"UR5n S U 0nOU[RR:XaUR S:wa[SUR S 35eS[R"UR50nUR(dSnOjS [R"UR50nOFU[RR :XaURnURnO[SUS35eUR#UUUUUS9$![a& [SURSUR35ef=f![a& [S URS UR35ef=f![a& [S URSUR35ef=f![a& [SURS UR35ef=f)afAdd an example (row) to a dataset from a run. Args: run (Run): The run to create an example from. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. created_at (Optional[datetime.datetime]): The creation timestamp of the example. Returns: Example: The created example Nrrz Run type z+ is not supported for dataset of type 'LLM'z2Error converting LLM run inputs to prompt for run z with inputs rjz7Error converting LLM run outputs to generation for run z with outputs rz, is not supported for dataset of type 'chat'z9Error converting LLM run inputs to chat messages for run z=Error converting LLM run outputs to chat generations for run z Dataset type z not recognized.rk)rr<rrbr_rrSrur;get_prompt_from_inputsrNrOget_llm_generation_from_outputschatget_messages_from_inputs#get_message_generation_from_outputsr`rm) rr`rrrl dataset_typerrNrOrns r?create_example_from_runClient.create_example_from_runs$  ** *EHHJL11*= :..22 2||u$  ~.11 !88D '.v%6F;;48!)!I!I#++!VJ $Z0 Z0055 5||u$  ~.22 !8#D#DSZZ#PQ ;;  ("N"NKK#GZ0033 3ZZFkkG}\N:JKL L""!%! #  g  x}SZZL: "$FF8>#++@  x}SZZL: "$$$'FF8>#++Hs08 G2 H"H8"I+0H0H580I(+0Jc /n0nU(a<[US[R5(d [S5eUSRnUGHn[U[R 5(dI[U[R5(d*[U[R 5(d [S5eURb[UR5nO[[R"55n[U[R 5(aSn O URn [U[R 5(a%URn URn URn OSupn 0U(aSW0O0EU bSU 0O0EU (aSU 0O0EU (aSU 0O0EU (aS U 0O0En URbURU S 'UR bUR U S '[#U 5nUR%USUS 0445 UR&b/[#UR&5nUR%US 3SUS 0445 UR(b/[#UR(5nUR%US3SUS 0445 UR*(GaUR*R-5Hunn[U[.5(a USnUSnOUunn[U[05(aU(a|[2R4R7U5n[9US5nUU[U5[[R"55-'UR%USU3SUUSU30445 M[S5eUR%USU3SUUS[AU530445 M [U[R 5(dGMURB(dGM[#URB5nUR%US3SUS 0445 GM [DRF"U[HS9nUR@S::aURK5nOUnUUU4$![:a [<R?SUU5 GMf=f)NrzWThe examples must be of type ExampleUpsertWithAttachments if include_dataset_id is Truez[The examples must be of type ExampleCreate or ExampleUpsertWithAttachments or ExampleUpdateNNNrrluse_source_run_iouse_source_run_attachments source_run_idrryrz.inputsz.outputs mime_typerr=z,Attachment file not found for example %s: %sz .attachment.z ; length=zMdangerously_allow_filesystem must be True to upload files from the filesystemz.attachments_operationsrr)&rqrbExampleUpsertWithAttachmentsrur ExampleCreate ExampleUpdater<rtrrrDrlrrrrryr'rxrNrOrrrr rXrxgetsizerEFileNotFoundErrorrrtrzattachments_operationsrrr&r)rrinclude_dataset_idrrrrrrrlrrr example_bodyvalbinputsboutputsbr1rrattachment_data file_sizer>attachments_operationsbrrs r?_prepare_multipart_dataClient._prepare_multipart_datas&(:< hqk:+R+RSS 5"!//JGw (@(@AA"7J,S,STT"7J,D,DEE ( zz% _  . ':#;#;<<! $// ':#;#;<<$+$=$=!-4-O-O* ' 5 5 PL!} 1CL*-1;1GL*-R ))*;< 223MN8EO]3"L++2+;+; Z(}}((/ W%|,D LL!l*  ~~)%gnn5 %,g. #.  *&w7 %,h/ $.  """(/(;(;(A(A(C$D*!*d33$.{$; *4V*<5?2 ?!/4887 ),.GGOOO,L '+OT'B!%. #O 4s4::<7H H"LL'1l,tf$E(,(,+4+Yyk(J(* %&!" #- o# #-,l4& A$($3'0k3;O:P$Q$& !" M)Df7J$<$<==222*5g6T6T*U' %,&=> 3.  k B!11%)L ;;* $$$&DD///}$5) &$R$.$3!" !) )s4+Q!Q98Q9)updatesrc$URUUUS9$)z{Update examples using multipart. .. deprecated:: 0.3.9 Use Client.update_examples instead. Will be removed in 0.4.0. rrr)_update_examples_multipart)rrrrs r?update_examples_multipart Client.update_examples_multipart%..!)E/  r>cURR=(d 0RSS5(d [S5eUc/nUR USUS9upEnUR S[ URU5U0URESUR0ES.S9n[R"U5 [[UR555 UR5$![[UR555 f=f) a Update examples using multipart. Args: dataset_id (Union[UUID, str]): The ID of the dataset to update. updates (Optional[List[ExampleUpdate]]): The updates to apply to the examples. Raises: ValueError: If the multipart examples endpoint is not enabled. "dataset_examples_multipart_enabledFzYour LangSmith deployment does not allow using the latest examples endpoints, please upgrade your deployment to the latest version or downgrade your SDK to langsmith<0.3.9.rrr8rrr$r=instance_flagsrrurr_dataset_examples_pathrrrr;rrrrr)rrrrrrrrs r?r!Client._update_examples_multiparts  ((.B33 0%  &  ?G+/+G+G $)E,H, ((  ;00&t||Z@  -- &(<(<  1 H  / / 9 /6689 :}} /6689 : AC$$$D)uploadsrc$URUUUS9$)z{Upload examples using multipart. .. deprecated:: 0.3.9 Use Client.create_examples instead. Will be removed in 0.4.0. rrr)_upload_examples_multipart)rrrrs r?upload_examples_multipart Client.upload_examples_multipartrr>cSnUR(a!U[[UR55- nUR(a!U[[UR55- nUR(a!U[[UR55- nUR (aUR R 5Hpup4[U[5(aUSnOUup5[U[5(a$U[RRU5- nOU[U5- nUS- nMr U$![[4a US- nN"f=f)z?Estimate the size of an example in bytes for batching purposes.rIrrT)rNrzr'rOrrrrqrr rXrxrrOSError)rrsizer.rrs r?_estimate_example_sizeClient._estimate_example_sizes >> C GNN34 4D ?? C GOO45 5D    C G$4$456 6D   !(!4!4!:!:!< j$//&0&8O)3&Aot44* @@C00D "= .w7* )*s>"D::EEcd/n/nSnUHnURU5nXr:a0U(aURU5 /nSnURU/5 MIXW-U:nU(aU(aURU5 U/nUnMvURU5 XW- nM U(aURU5 U$)zBatch examples by size limits.r)rrx) rrrBbatches current_batch current_sizer example_size size_exceededs r?_batch_examples_by_sizeClient._batch_examples_by_size.s 8:  G66w?L2 NN=1$&M#$Ly)(7:NNM}-!( + $$W-, / 4  NN= )r>cURR=(d 0RSS5(d [S5eUc/nUR USUS9upEnUR S[ URU5U0URESUR0ES.S9n[R"U5 [[UR555 UR5$![[UR555 f=f) aUpload examples using multipart. Args: dataset_id (Union[UUID, str]): The ID of the dataset to upload to. uploads (Optional[List[ExampleCreate]]): The examples to upload. dangerously_allow_filesystem (bool): Whether to allow uploading files from the filesystem. Returns: ls_schemas.UpsertExamplesResponse: The count and ids of the successfully uploaded examples Raises: ValueError: If the multipart examples endpoint is not enabled. rFYour LangSmith deployment does not allow using the multipart examples endpoint, please upgrade your deployment to the latest version.rrrrr$r)rrrrrrrrs r?r!Client._upload_examples_multipartWs( ((.B33 0%  X  ?G+/+G+G $)E,H, ((  ;00&t||Z@  -- &(<(<  1 H  / / 9 /6689 :}} /6689 :r)upsertsrc>URR=(d 0RSS5(d [S5eUc/nUR USUS9up4nUR SUR SSS :waUR S SS :waS OS U0URESUR0ES.S9n[R"U5 [[UR555 UR5$![[UR555 f=f)zUpsert examples. .. deprecated:: 0.3.9 Use Client.create_examples and Client.update_examples instead. Will be removed in 0.4.0. examples_multipart_enabledFrNTrr/v1z/v1/z/v1/platform/examples/multipartz/platform/examples/multipartrrr$)r=rrrurrrrrr;rrrrr)rrrrrrrs r?upsert_examples_multipart Client.upsert_examples_multiparts6 ((.B33 (%  X  ?G+/+G+G #)E,H, ((  ;00||BC(E1dll236G66Q67! -- &(<(<  1H  / / 9 /6689 :}} /6689 :s A,C88$D)rrrrmax_concurrencyc PSUs=::aS::d O [S5eU(a<U(a5SRUVs/sH nSUS3PM sn5n[SUS35e1Skn U(a9[U5RU 5=n (a[S [ U 5S 35eU(dU(d [S 5eU(dUR US 9R nU(a?UV s/sH0n [U [5(a[R"S 0U D6OU PM2 n n GOURS 5n U (d [S5e[U 5nUR5H8unnUcM [U5U:wdM[SUS[U5SUS35e [U URS5=(d S[U55URS5=(d S[U55URS5=(d S[U55URS5=(d S[U55URS5=(d S[U555VVVVVVs/sHLunnnnnn[R"S 0UUUUU=(d [!["R$"55US.D6PMN n nnnnnnU (d[R&"/SS9$UR)U 5nUR+UX$U5$s snfs sn fs snnnnnnf)!aH Create examples in a dataset. Args: dataset_name (str | None): The name of the dataset to create the examples in. Must specify exactly one of dataset_name or dataset_id. dataset_id (UUID | str | None): The ID of the dataset to create the examples in. Must specify exactly one of dataset_name or dataset_id examples (Sequence[ExampleCreate | dict]): The examples to create. dangerously_allow_filesystem (bool): Whether to allow uploading files from the filesystem. **kwargs (Any): Legacy keyword args. Should not be specified if 'examples' is specified. - inputs (Sequence[Mapping[str, Any]]): The input values for the examples. - outputs (Optional[Sequence[Optional[Mapping[str, Any]]]]): The output values for the examples. - metadata (Optional[Sequence[Optional[Mapping[str, Any]]]]): The metadata for the examples. - splits (Optional[Sequence[Optional[str | List[str]]]]): The splits for the examples, which are divisions of your dataset such as 'train', 'test', or 'validation'. - source_run_ids (Optional[Sequence[Optional[Union[UUID, str]]]]): The IDs of the source runs associated with the examples. - ids (Optional[Sequence[Union[UUID, str]]]): The IDs of the examples. Raises: ValueError: If 'examples' and legacy args are both provided. Returns: The LangSmith JSON response. Includes 'count' and 'example_ids'. !!! warning "Behavior changed in `langsmith` 0.3.11" Updated to take argument 'examples', a single list where each element is the full example to create. This should be used instead of the legacy 'inputs', 'outputs', etc. arguments which split each examples attributes across arguments. Updated to support creating examples with attachments. Example: ```python from langsmith import Client client = Client() dataset = client.create_dataset("agent-qa") examples = [ { "inputs": {"question": "what's an agent"}, "outputs": {"answer": "an agent is..."}, "metadata": {"difficulty": "easy"}, }, { "inputs": { "question": "can you explain the agent architecture in this diagram?" }, "outputs": {"answer": "this diagram shows..."}, "attachments": {"diagram": {"mime_type": "image/png", "data": b"..."}}, "metadata": {"difficulty": "medium"}, }, # more examples... ] response = client.create_examples(dataset_name="agent-qa", examples=examples) # -> {"example_ids": [... ``` rrz'max_concurrency must be between 1 and 3z, rzCannot specify z when 'examples' is specified.>idsrNsplitsrOrsource_run_idsz(Received unsupported keyword arguments: rz3Either dataset_id or dataset_name must be provided.rrNz+Must specify either 'examples' or 'inputs.' Length of  (z#) does not match length of inputs (rrOc3&# UHnSv M g7frr=rr.s r?r)Client.create_examples..Cs-MJ!*-KL   c&k&<&<=M&NN{N:5;M:NaP lRS S** *EHHJ "!A20$sL&7L:ALc /nSnSSKJn U"US9nUV s/sH!n URURU UU5PM# n n [R "U 5HHn U R 5n URU RS/55 XlRSS5- nMJ SSS5 [R"XVS9$s sn f!,(df  N'=f)Nr)ContextThreadPoolExecutor max_workersrrr) langsmith.utilsrr_upload_single_batchr=rrJr|rrbr) rrrrrall_examples_ids total_countrrrrArrs r?r(Client._upload_examples_batches_parallelUs = &? Cx%%E --0  % //'2!==? '' ]B(GH||GQ77 3D"00(  D CsC(B>A$C>C CcURR=(d 0RSS5(aNUR[ [ R U5UUS9nURS/5URSS5S.$UH/n[US5cMS Ul[R"S 5 M1 URS S 0URES S0E[UVs/sHn0[USS9ES[U50EPM sn5S9n[ R""U5 UR%5nUVs/sHowSPM sn['U5S.$s snfs snf)zIUpload a single batch of examples (used by both sequential and parallel).rFrrrrrrNz7Must upgrade your LangSmith version to use attachments.r/examples/bulkrrT exclude_nonerrr<)r=rrrrrrrsrVrrrrrr' dump_modelrtr;rrrz)rrrrrupload response_datars r?rClient._upload_single_batchqso II $ $ * / / 0%  66 :6-I7H (||M2>!gq1   6=1=)-F&MMQ 00 M4==M.:LM ',  ',F (dC(#j/',  1 H  / / 9$MMOM7DE}tT }E]+  Fs #E5Ec UcU (d [S5eUcURUS9Rn[R"S0UUUUU U U U S.D6n U(aXMlU(a-[ U[5(a[R"U5OUO[R"5U lURR=(d 0RSS5(a)URX-/S9 URU RS9$[!U S 5bSU l[$R&"S 5 UR)S S 0UR*ES S0E[-0[/U 5R15VVs0sH upUcM X_M snnES[U50E5S9n[2R4"U5 UR75n[R8"S0UDUR:UR=5S.D6$s snnf)aCreate a dataset example in the LangSmith API. Examples are rows in a dataset, containing the inputs and expected outputs (or other reference information) for a model or chain. Args: inputs (Mapping[str, Any]): The input values for the example. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to create the example in. dataset_name (Optional[str]): The name of the dataset to create the example in. created_at (Optional[datetime.datetime]): The creation timestamp of the example. outputs (Optional[Mapping[str, Any]]): The output values for the example. metadata (Optional[Mapping[str, Any]]): The metadata for the example. split (Optional[str | List[str]]): The splits for the example, which are divisions of your dataset such as 'train', 'test', or 'validation'. example_id (Optional[Union[UUID, str]]): The ID of the example to create. If not provided, a new example will be created. source_run_id (Optional[Union[UUID, str]]): The ID of the source run associated with this example. use_source_run_io (bool): Whether to use the inputs, outputs, and attachments from the source run. use_source_run_attachments (Optional[List[str]]): Which attachments to use from the source run. If use_source_run_io is True, all attachments will be used regardless of this param. attachments (Optional[Attachments]): The attachments for the example. Returns: Example: The created example. Nz/Must provide either inputs or use_source_run_ior)rNrOrryrrrrrF)rr)rrz6Must upgrade your LangSmith version to use attachmentsrrrrrrrBr=)rurr<rbrrlrqrtrrrsrDr=rrr read_examplerVrrrrrr'rrr;rrrrrG)rrNrrrlrOrryrrrrrrrmrnrrJs r?rmClient.create_examplesl >"3NO O  ** *EHHJ''   "$!.%6.H*    (O'1S&A&ATYYz "z  II $ $ * / / 0%    + +z6 + R$$$8 8t]+7#'  VW00M4==M.:LM ,6t,<,B,B,DV,DDA414,DV$c*o 1 H  / / 9]]_F%%..779 Ws 3 G7G7)rOc URSS[US53SU(aUR5OS0S9nUR5n[ USUR S9n[ R"S 0UR5VVs0sHupgUS:wdM Xg_M snnDUURUR5S .D6$s snnf) aRead an example from the LangSmith API. Args: example_id (Union[UUID, str]): The ID of the example to read. as_of (Optional[datetime.datetime]): The dataset version tag OR timestamp to retrieve the example as of. Response examples will only be those that were present at the time of the tagged (or timestamped) version. Returns: Example: The example. r /examples/rrONrattachment_urlsr\rrrr=) rr{r,rr^rrbrrrrG)rrrOrrrrmrns r?rClient.read_example s,, *l;< =e*- --/E %6  !! ' J18I3ItqtJ #nn335   Js  B<B<r)rrrreinclude_attachmentsc +# 0U EUU[U[R5(aUR5OUUUUb [US5OSU S.En U b[ U 5U S'UbXS'O"UbUR US9R nXS'OU (a/SQU S'[URS U S 95Hup[US URS 9n[R"S0UR5VVs0sHunnUS :wdMUU_M snnDUURUR5S .D6v UcMUS-U:dM g gs snnf7f)a Retrieve the example rows of the specified dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to filter by. dataset_name (Optional[str]): The name of the dataset to filter by. example_ids (Optional[Sequence[Union[UUID, str]]): The IDs of the examples to filter by. as_of (Optional[Union[datetime.datetime, str]]): The dataset version tag OR timestamp to retrieve the examples as of. Response examples will only be those that were present at the time of the tagged (or timestamped) version. splits (Optional[Sequence[str]]): A list of dataset splits, which are divisions of your dataset such as 'train', 'test', or 'validation'. Returns examples only from the specified splits. inline_s3_urls (bool, default=True): Whether to inline S3 URLs. offset (int, default=0): The offset to start from. Defaults to 0. limit (Optional[int]): The maximum number of examples to return. metadata (Optional[dict]): A dictionary of metadata to filter by. filter (Optional[str]): A structured filter string to apply to the examples. include_attachments (bool, default=False): Whether to include the attachments in the response. **kwargs (Any): Additional keyword arguments are ignored. Yields: The examples. Examples: List all examples for a dataset: ```python from langsmith import Client client = Client() # By Dataset ID examples = client.list_examples( dataset_id="c9ace0d8-a82c-4b6c-13d2-83401d68e9ab" ) # By Dataset Name examples = client.list_examples(dataset_name="My Test Dataset") ``` List examples by id ```python example_ids = [ "734fc6a0-c187-4266-9721-90b7a025751a", "d6b4c1b9-6160-4d63-9b61-b034c585074f", "4d31df4e-f9c3-4a6e-8b6c-65701c2fed13", ] examples = client.list_examples(example_ids=example_ids) ``` List examples by metadata ```python examples = client.list_examples( dataset_name=dataset_name, metadata={"foo": "bar"} ) ``` List examples by structured filter ```python examples = client.list_examples( dataset_name=dataset_name, filter='and(not(has(metadata, \'{"foo": "bar"}\')), exists(metadata, "tenant_id"))', ) ``` NrU)rr<rOrinline_s3_urlsrrerr,r)rrOrrirrrr\rrr=)rqr[r,rr'rr<rrr^rrbrrrrG)rrrrrOrrrrrrerrrrrrrmrns r?rClient.list_examples-sjl "  " %/x7H7H%I%I!u,(-(9S_s "   !,X!6F:   ! *9   %** *EHHJ *9   IF8 #  $ $[ $ @ JAJ):DLLK$$$+MMONODAqq E latest)rPc [US5nURSSUS3UR[R"SU0UE5S9n[ R "U5 g)aEnable dataset indexing. Examples are indexed by their inputs. This enables searching for similar examples by inputs with ``client.similar_examples()``. Args: dataset_id (Union[UUID, str]): The ID of the dataset to index. tag (Optional[str]): The version of the dataset to index. If 'latest' then any updates to the dataset (additions, updates, deletions of examples) will be reflected in the index. **kwargs (Any): Additional keyword arguments to pass as part of request body. Returns: None rrrz/indexrPrNr{rrrrr;r)rrrPrresps r? index_datasetClient.index_datasetsa.j,7 ((  F +MMUC2623 )  ++D1r>c [US5nURSSUS3UR[R"0UE5S9n[ R "U5 g)zSync dataset index. This already happens automatically every 5 minutes, but you can call this to force a sync. Args: dataset_id (Union[UUID, str]): The ID of the dataset to sync. Returns: None rrrz /index/syncrNr )rrrr s r?sync_indexed_datasetClient.sync_indexed_datasets[$j,7 ((  K 0MMJvJ' )  ++D1r>)rec X[US5nUUS.UEnUbXFS'URSSUS3UR[R"U5S9n[ R "U5 /nUR5SH,n UR[R"S 0U DSU0D65 M. U$) aRetrieve the dataset examples whose inputs best match the current inputs. !!! note Must have few-shot indexing enabled for the dataset. See `client.index_dataset()`. Args: inputs (dict): The inputs to use as a search query. Must match the dataset input schema. Must be JSON serializable. limit (int): The maximum number of examples to return. dataset_id (Union[UUID, str]): The ID of the dataset to search over. filter (Optional[str]): A filter string to apply to the search results. Uses the same syntax as the `filter` parameter in `list_runs()`. Only a subset of operations are supported. For example, you can use ``and(eq(metadata.some_tag, 'some_value'), neq(metadata.env, 'dev'))`` to filter only examples where some_tag has some_value, and the environment is not dev. **kwargs: Additional keyword arguments to pass as part of request body. Returns: list[ExampleSearch]: List of ExampleSearch objects. Examples: ```python from langsmith import Client client = Client() client.similar_examples( {"question": "When would i use the runnable generator"}, limit=3, dataset_id="...", ) ``` ```python [ ExampleSearch( inputs={ "question": "How do I cache a Chat model? What caches can I use?" }, outputs={ "answer": "You can use LangChain's caching layer for Chat Models. This can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times, and speed up your application.\n\nfrom langchain.cache import InMemoryCache\nlangchain.llm_cache = InMemoryCache()\n\n# The first time, it is not yet in cache, so it should take longer\nllm.predict('Tell me a joke')\n\nYou can also use SQLite Cache which uses a SQLite database:\n\nrm .langchain.db\n\nfrom langchain.cache import SQLiteCache\nlangchain.llm_cache = SQLiteCache(database_path=\".langchain.db\")\n\n# The first time, it is not yet in cache, so it should take longer\nllm.predict('Tell me a joke') \n" }, metadata=None, id=UUID("b2ddd1c4-dff6-49ae-8544-f48e39053398"), dataset_id=UUID("01b6ce0f-bfb6-4f48-bbb8-f19272135d40"), ), ExampleSearch( inputs={"question": "What's a runnable lambda?"}, outputs={ "answer": "A runnable lambda is an object that implements LangChain's `Runnable` interface and runs a callbale (i.e., a function). Note the function must accept a single argument." }, metadata=None, id=UUID("f94104a7-2434-4ba7-8293-6a283f4860b4"), dataset_id=UUID("01b6ce0f-bfb6-4f48-bbb8-f19272135d40"), ), ExampleSearch( inputs={"question": "Show me how to use RecursiveURLLoader"}, outputs={ "answer": 'The RecursiveURLLoader comes from the langchain.document_loaders.recursive_url_loader module. Here\'s an example of how to use it:\n\nfrom langchain.document_loaders.recursive_url_loader import RecursiveUrlLoader\n\n# Create an instance of RecursiveUrlLoader with the URL you want to load\nloader = RecursiveUrlLoader(url="https://example.com")\n\n# Load all child links from the URL page\nchild_links = loader.load()\n\n# Print the child links\nfor link in child_links:\n print(link)\n\nMake sure to replace "https://example.com" with the actual URL you want to load. The load() method returns a list of child links found on the URL page. You can iterate over this list to access each child link.' }, metadata=None, id=UUID("0308ea70-a803-4181-a37d-39e95f138f8c"), dataset_id=UUID("01b6ce0f-bfb6-4f48-bbb8-f19272135d40"), ), ] ``` r)rNrrerrz/searchrrr=) r{rrrrr;rrxrb ExampleSearch) rrNrrrerreqr rexs r?similar_examplesClient.similar_exampless^j,7     "M((  G ,MMC )  ++D1))+j)B OOJ44QrQjQ R*r>)rNrOrryrrrcJUb?URR=(d 0RSS5(d [S5e[ UUUUUUUS9n [ R "S 0U R5V V s0sH upU cM X_M sn n D6n UcURU5RnURR=(d 0RSS5(a[ URXl/S95$URSS[US530URES S 0E[0[U 5R5V V s0sH upU cM X_M sn n ES [!U50E5S 9n ["R$"U 5 U R'5$s sn n fs sn n f)aUpdate a specific example. Args: example_id (Union[UUID, str]): The ID of the example to update. inputs (Optional[Dict[str, Any]]): The input values to update. outputs (Optional[Mapping[str, Any]]): The output values to update. metadata (Optional[Dict]): The metadata to update. split (Optional[str | List[str]]): The dataset split to update, such as 'train', 'test', or 'validation'. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to update. attachments_operations (Optional[AttachmentsOperations]): The attachments operations to perform. attachments (Optional[Attachments]): The attachments to add to the example. Returns: Dict[str, Any]: The updated example. rFYour LangSmith deployment does not allow using the attachment operations, please upgrade your deployment to the latest version.)rNrOr<rryrr)rrr8rrrrrrr=)r=rrrurrbrrrrrrr{rr'rrtr;rr)rrrNrOrryrrr example_dictrmrnrrs r?update_exampleClient.update_exampleJsH " -II,,2774e!V#9# ** , 2 2 4F 4tqt 4F   **:6AAJ II $ $ * / / 0%  //)90 00Xj,?@AM4==M.:LM )37(;(A(A(C(C !AD(C %c*o   1H  / / 9==? "AG,s7 FF FF)rrrrc b U(aU(a[S[U5S35eU(dU(d [S5eU(aU(a [S5eU(aURUS9RnU(avUVs/sH0n[ U[ 5(a[ R"S-0UD6OUPM2 nnU(d0USR(aUSRnGO[S5eGOURSS 5nU(d [S 5eU(dkS U;aURUS5RnOF[[US 55S :a [S 5eUS S(d [S5eUS SnURR=(d 0RS5n U (d6URS5=(d URS5b [S5e[U5n UR5H7upU cM [U 5U :wdM[SU S[U 5SU S35e [!UURSS[#U 555URSS[#U 555URSS[#U 555URSS[#U 555URS S[#U 555URSS [#U 555URSS![#U 5555V VVVVVVVs /sH*upnnnnnn[ R"S-0U UUUUUUUS".D6PM, nnnnnnnn nS nURR=(d 0RSS#5(aBUR%['[(R*U5UUS$9nS%URS&S5S'30UE$UR-S(S)0UR.ES*S+0E[1UVVVs/sH8n[3U5R5VVs0sHunnUcM UU_M snnPM: snnn5S,9n[4R6"U5 UR95$s snfs s nnnnnnnn fs snnfs snnnf).a Update multiple examples. Examples are expected to all be part of the same dataset. Args: dataset_name (str | None): The name of the dataset to update. Should specify exactly one of 'dataset_name' or 'dataset_id'. dataset_id (UUID | str | None): The ID of the dataset to update. Should specify exactly one of 'dataset_name' or 'dataset_id'. updates (Sequence[ExampleUpdate | dict] | None): The example updates. Overwrites any specified fields and does not update any unspecified fields. dangerously_allow_filesystem (bool): Whether to allow using filesystem paths as attachments. **kwargs (Any): Legacy keyword args. Should not be specified if 'updates' is specified. - example_ids (Sequence[UUID | str]): The IDs of the examples to update. - inputs (Sequence[dict | None] | None): The input values for the examples. - outputs (Sequence[dict | None] | None): The output values for the examples. - metadata (Sequence[dict | None] | None): The metadata for the examples. - splits (Sequence[str | list[str] | None] | None): The splits for the examples, which are divisions of your dataset such as 'train', 'test', or 'validation'. - attachments_operations (Sequence[AttachmentsOperations | None] | None): The operations to perform on the attachments. - dataset_ids (Sequence[UUID | str] | None): The IDs of the datasets to move the examples to. Returns: The LangSmith JSON response. Includes 'message', 'count', and 'example_ids'. !!! warning "Behavior changed in `langsmith` 0.3.9" Updated to ... Example: ```python from langsmith import Client client = Client() dataset = client.create_dataset("agent-qa") examples = [ { "inputs": {"question": "what's an agent"}, "outputs": {"answer": "an agent is..."}, "metadata": {"difficulty": "easy"}, }, { "inputs": { "question": "can you explain the agent architecture in this diagram?" }, "outputs": {"answer": "this diagram shows..."}, "attachments": {"diagram": {"mime_type": "image/png", "data": b"..."}}, "metadata": {"difficulty": "medium"}, }, # more examples... ] response = client.create_examples(dataset_name="agent-qa", examples=examples) example_ids = response["example_ids"] updates = [ { "id": example_ids[0], "inputs": {"question": "what isn't an agent"}, "outputs": {"answer": "an agent is not..."}, }, { "id": example_ids[1], "attachments_operations": [ {"rename": {"diagram": "agent_diagram"}, "retain": []} ], }, ] response = client.update_examples(dataset_name="agent-qa", updates=updates) # -> {"example_ids": [... ``` z&Must pass in either 'updates' or args z , not both.z6Please pass in a non-empty sequence for arg 'updates'.z;Must pass in exactly one of 'dataset_name' or 'dataset_id'.rrz=Must pass in (exactly) one of 'dataset_name' or 'dataset_id'.rNz9Must pass in (exactly) one of 'updates' or 'example_ids'.rErz-Dataset IDs must be the same for all examplesz+If specified, dataset_ids must be non-null.rrrrrrz%) does not match length of examples (rrNc3&# UHnSv M g7frr=rs r?r)Client.update_examples..J)L8K1$8KrrOc3&# UHnSv M g7frr=rs r?rrKs*M9LA49Lrrc3&# UHnSv M g7frr=rs r?rrLs+N:MQD:Mrrc3&# UHnSv M g7frr=rs r?rrMr rc3&# UHnSv M g7frr=rs r?rrN.Q=Pt=Prc3&# UHnSv M g7frr=rs r?rrOr%rc3&# UHnSv M g7frr=rs r?rrQs2UATA4ATr)r<rNrOrrryrrFrrxrz examples updatedr8rrrrr=)rurrr<rqrrbrrrrrzrZr=rrrrrrrrrsrrr'rr;rr)rrrrrrr updates_objrmultipart_enabled examples_lenrrrrrrr dataset_id_ attachments_attachments_operations_rrrmrns r?update_examplesClient.update_examplessp g8v{S 'UV V JM ** *EHHJ ! A2,,!,Q!:!:J$W !**]D9K O .!%!2!2;q>!B!M!MJVM234q8$%TUU .q1$%RSS!' !6q!9J!%!9!9!?R D D4! &ZZ 89VVZZ =V!X {+L'-||~#(S^|-K$$XJbY0@A00<~Q@(6&nqJJx)Ll8K)LMJJy*M|9L*MNJJz+N% :M+NOJJx)Ll8K)LMJJ}.QU<=P.QRJJ}.QU<=P.QRJJ02U|AT2U n niCdIv{LRi(( !"%#'&1$-!''32I   n 6 II $ $ * / / 0%  66 :6#-I7Hhll7A677HI  00 M4==M.:LM,7  ,7-7w,?,E,E,G,GDAq#$!%1,G ,7   1H"  / / 9==? "afds*7R1R#"R* R$R$R*$R*cURSS[US53URS9n[R"U5 g)zDelete an example by ID. Args: example_id (Union[UUID, str]): The ID of the example to delete. Returns: None rrrrNr)rrrs r?delete_exampleClient.delete_examplezsE,, *l;< =MM-  ++H5r>) hard_deletec ZU(a[U5VVs/sHup4[[USUS355PM snnSS.n[URS5n[ URU5nUR RSU0URESS0E[U5URS 9nO[S [U5VVs/sHup4[[USUS355PM snn0n URS S 0URESS0EU S 9n[R"U5 gs snnfs snnf)zDelete multiple examples by ID. Parameters ---------- example_ids : Sequence[ID_TYPE] The IDs of the examples to delete. hard_delete : bool, default=False If True, permanently delete the examples. If False, soft delete them. example_ids[]T)rr3zdatasets/examples/deleterrr)rCrrrrrr;N)rrtr{_platform_pathrrrrrr'rrr;r) rrr3rrr rxfull_urlrrs r?delete_examplesClient.delete_examplessB #,K"8 "8 QCq&9:;"8  $ D"$,,0JKD%dllD9H||++M4==M.:LM & ,H"+K"8 "8 QCq&9:;"8 &F 00M4==M.:LM 1H ++H5= & s %D!>%D')rrrOcRUc'Uc [S5eURUS9Rn0nUb4[U[R5(aUR 5OUUS'UR SS[US5S3US9n[R"U5 UR5$) aGet the splits for a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. as_of (Optional[Union[str, datetime.datetime]]): The version of the dataset to retrieve splits for. Can be a timestamp or a string tag. Defaults to "latest". Returns: List[str]: The names of this dataset's splits. Must provide dataset name or IDrrOrrr/splitsr) rurr<rqr[r,rr{r;rr)rrrrOrrs r?list_dataset_splitsClient.list_dataset_splitss&  # !BCC** *EHHJ  %/x7H7H%I%I!u 7O,, *l;)rrryc DUc'Uc [S5eURUS9RnU[U5VVs/sHupg[ [ USUS355PM snnUS.nUR SS[ US 5S 3US 9n [R"U 5 gs snnf) aUpdate the splits for a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to update. dataset_name (Optional[str]): The name of the dataset to update. split_name (str): The name of the split to update. example_ids (List[Union[UUID, str]]): The IDs of the examples to add to or remove from the split. remove (Optional[bool]): If True, remove the examples from the split. If False, add the examples to the split. Returns: None Nr<rr5r6) split_namerryrrrr=r) rurr<rrtr{rr;r) rrrrArryrrrrs r?update_dataset_splitsClient.update_dataset_splitss.  # !BCC** *EHHJ$( 44FAHSL1"5674  ,, Z\ BC7KRV-  ++H5s%Bc[U[[R45(aUR XS9nU$[ [ RU5nU$)zResolve the run ID. Args: run (Union[Run, RunBase, str, UUID]): The run to resolve. load_child_runs (bool): Whether to load child runs. Returns: Run: The resolved run. Raises: TypeError: If the run type is invalid. r`)rqrtrrrsrarrbr_)rr`r`run_s r?_resolve_run_idClient._resolve_run_id sH& cC+ , ,===FD  ,D r>c[U[[R45(aUR U5nU$[U[ R 5(aUnU$[U[5(a5[ R "S0UDURUR5S.D6nU$URbUR UR5nU$SnU$)zResolve the example ID. Args: example (Optional[Union[Example, str, UUID, dict]]): The example to resolve. run (Run): The run associated with the example. Returns: Optional[Example]: The resolved example. rBNr=) rqrtrrrsrrbrrrrGr')rrr`reference_example_s r?_resolve_example_idClient._resolve_example_id#s gTYY/ 0 0!%!2!27!; "!!3!3 4 4!( "! & &!+!3!3""..779" "!  % % 1!%!2!233K3K!L "!"& !!r>)fn_namec^^^SSKJm SUU4SjjnS Sjn[UTR5(aU/nU$U"U5(aUSVs/sH oc"U5PM nnU$[U[5(aU"[ [U55/nU$[ S[U5S35es snf) Nrr]c~>[U[5(a&TR"S0TURS5S.UED6$U$)N reasoning)rcommentr=)rqrEvaluationResultr) single_resultrN ls_evaluators r? _cast_result1Client._select_eval_results.._cast_resultMsR-..#44�#4#4[#A(! r>rc:[U[5=(a SU;$)Nr)rqr)rs r?_is_eval_results5Client._select_eval_results.._is_eval_resultsZsgt,Eg1E Er>z!Invalid evaluation results type: z.. Must be EvaluationResult, EvaluationResults.)rTz*Union[ls_evaluator.EvaluationResult, dict]rls_evaluator.EvaluationResult)rrrz)TypeGuard[ls_evaluator.EvaluationResults])langsmith.evaluationr^rqrSrrrur)rrrNrVrYresults_rKrUs ` @r?_select_eval_resultsClient._select_eval_resultsCs C !E ! * ! ! F g|<< = =yHg & &181CD1CA Q1CHD & &$T$%89:H  3DM?C@@  EsB*) source_inforxr`cURXS9nURXF5nURUUS9nURUUUS9n U S$)aEvaluate a run. Args: run (Union[Run, RunBase, str, UUID]): The run to evaluate. evaluator (RunEvaluator): The evaluator to use. source_info (Optional[Dict[str, Any]]): Additional information about the source of the evaluation to log as feedback metadata. reference_example (Optional[Union[Example, str, dict, UUID]]): The example to use as a reference for the evaluation. If not provided, the run's reference example will be used. load_child_runs (bool, default=False): Whether to load child runs when resolving the run ID. Returns: Feedback: The feedback object created by the evaluation. rFrr`r)rHrL evaluate_run_log_evaluation_feedback rr`r^r`rxr`rGrKevaluator_responsers r?rdClient.evaluate_runjsq<##C#I!556GN&33 &4 //  #0  qzr>) _executorc^^TRU5nUU4SjnUGH7nU=(d 0n UR(a0UREU En Sn UR(a URn OUb URn URbURR SS5OSn U"U UR URURURURU UR[[[RUR 5[R"R$UURU(a UR&OSU S9 GM: U$)Nct>T(aTR"TR40UD6 gTR"S0UD6 g)Nr=)rcreate_feedback)rrirs r?_submit_feedback9Client._log_evaluation_feedback.._submit_feedbacks0  !5!5@@$$.v.r>r)r)rrrrR correctionr`rfeedback_configfeedback_source_typercrPrwr)r^evaluator_info target_run_idr<rPrrrrrRrorrrrbFeedbackConfigrpFeedbackSourceTypeMODELrw) rrgr`r`rcrirrmres source_info_rrs ` ` r?reClient._log_evaluation_feedbacks++,>? / C&,"L!!E#"4"4E E G  ++&&47II4ICIIMM'40tE GGiiii >>(!// $Z6679L9L!&0%B%B%H%H%ii),$! :r>c# URXS9nURXF5nURUUS9IShvN nURUUUS9n U S$N7f)aEvaluate a run asynchronously. Args: run (Union[Run, str, UUID]): The run to evaluate. evaluator (RunEvaluator): The evaluator to use. source_info (Optional[Dict[str, Any]]): Additional information about the source of the evaluation to log as feedback metadata. reference_example (Optional[Union[Example, str, dict, UUID]]): The example to use as a reference for the evaluation. If not provided, the run's reference example will be used. load_child_runs (bool, default=False): Whether to load child runs when resolving the run ID. Returns: EvaluationResult: The evaluation result object created by the evaluation. rFrbNrcr)rHrL aevaluate_runrerfs r?r{Client.aevaluate_runs~<##C#I!556GN#,#:#: &$;$   //  #0  qz s4AAArN)rrrwrorRr`rqr feedback_idrprrccomparative_experiment_idfeedback_group_idrPrc x U=(d UnUcUc [S5eUbUb [S5eU(a'[R"S[U53[5 [ U [ R5(d[ R"U 5n U [ RR:Xa[ R"US9nOAU [ RR:Xa[ R"US9nO[SU 35eURb URO0Ul U b*SUR;aS[U 50URS'UR(aSUR;a}URSn[US 5(a [U5(aUR!5nSU;a([[#URSSS 55US'UURS'[ R$"S'0S ['U 5_S['US S 9_S['US S 9_SU_S[)U5_SU_SU_SU_SU_S[*R*R-[*R.R05_S[*R*R-[*R.R05_SU _S['US S 9_S['US S 9_S['US S 9_SU_SU_6nUR2(+=(a/ UR4R6=(d 0R9SS 5nU(GaxUR4R:(Ga\[<R>"UR4R:S5(Ga+UR@cURBGbURDGbURFc[IU5nURBb[KU5nURBRL [OUURB[P5nU(aJURB=RRS- sl)URT(aURTRW5 SSS5 OUR@b8UR@RY[[[UR\5U55 O<[_UR!S S 95nURaS!S"S#U0U [<Rb4S$9 [ Rd"S'0UR!5D6$!,(df  N2=f![fan[hRkS%S S&9 UeSnAff=f)(a:Create feedback for a run. !!! note To enable feedback to be batch uploaded in the background you must specify `trace_id`. *We highly encourage this for latency-sensitive environments.* Args: key (str): The name of the feedback metric. score (Optional[Union[float, int, bool]]): The score to rate this run on the metric or aspect. value (Optional[Union[float, int, bool, str, dict]]): The display value or non-numeric value for this feedback. run_id (Optional[Union[UUID, str]]): The ID of the run to provide feedback for. At least one of run_id, trace_id, or project_id must be specified. trace_id (Optional[Union[UUID, str]]): The ID of the trace (i.e. root parent run) of the run to provide feedback for (specified by run_id). If run_id and trace_id are the same, only trace_id needs to be specified. **NOTE**: trace_id is required feedback ingestion to be batched and backgrounded. correction (Optional[dict]): The proper ground truth for this run. comment (Optional[str]): A comment about this feedback, such as a justification for the score or chain-of-thought trajectory for an LLM judge. source_info (Optional[Dict[str, Any]]): Information about the source of this feedback. feedback_source_type (Union[FeedbackSourceType, str]): The type of feedback source, such as model (for model-generated feedback) or API. source_run_id (Optional[Union[UUID, str]]): The ID of the run that generated this feedback, if a "model" type. feedback_id (Optional[Union[UUID, str]]): The ID of the feedback to create. If not provided, a random UUID will be generated. feedback_config (Optional[FeedbackConfig]): The configuration specifying how to interpret feedback with this key. Examples include continuous (with min/max bounds), categorical, or freeform. stop_after_attempt (int, default=10): The number of times to retry the request before giving up. project_id (Optional[Union[UUID, str]]): The ID of the project (or experiment) to provide feedback on. This is used for creating summary metrics for experiments. Cannot specify run_id or trace_id if project_id is specified, and vice versa. comparative_experiment_id (Optional[Union[UUID, str]]): If this feedback was logged as a part of a comparative experiment, this associates the feedback with that experiment. feedback_group_id (Optional[Union[UUID, str]]): When logging preferences, ranking runs, or other comparative feedback, this is used to group feedback together. extra (Optional[Dict]): Metadata for the feedback. **kwargs (Any): Additional keyword arguments. Returns: Feedback: The created feedback object. Example: ```python from langsmith import trace, traceable, Client @traceable def foo(x): return {"y": x * 2} @traceable def bar(y): return {"z": y - 1} client = Client() inputs = {"x": 1} with trace(name="foobar", inputs=inputs) as root_run: result = foo(**inputs) result = bar(**result) root_run.outputs = result trace_id = root_run.id child_runs = root_run.child_runs # Provide feedback for a trace (a.k.a. a root run) client.create_feedback( key="user_feedback", score=1, trace_id=trace_id, ) # Provide feedback for a child run foo_run_id = [run for run in child_runs if run.name == "foo"][0].id client.create_feedback( key="correctness", score=0, run_id=foo_run_id, # trace_id= is optional but recommended to enable batched and backgrounded # feedback ingestion. trace_id=trace_id, ) ``` Nz8One of run_id, trace_id, or project_id must be providedz?project_id cannot be provided if run_id or trace_id is providedzLThe following arguments are no longer used in the create_feedback endpoint: )rzUnknown feedback source type __runr)rz+feedback_source.metadata['__run']['run_id']r<TrrwrrrrorRfeedback_sourcerl modified_atrpr+r~rrPruse_multipart_endpointz0.8.10rrr /feedbackr)rrrzError creating feedbackrr=)6rurrrPr~rqrbruAPIAPIFeedbackSourcervModelFeedbackSourcerrtrzrUrr{FeedbackCreaterrr[r\r]r^r0r=rrrr;is_version_greater_or_equalrr&rwr+r/r1rr.r&rr'rZrr r<r'rrFeedbackrrr)rr)rrrrwrorRr`rqrr}rprrcr~rrPrrr _run_metafeedbackr/rrrfeedback_blockrs r?rlClient.create_feedbacksF#8 >j0WX X  *"8Q   MM$Vn-/"  i 2J4Q4QRR'1'D'D(($$z'D'D'H'HH00+F &)F)F)L)LL","@"@+"V #@AU@V!WXX,;,D,D,P((VX  $(WO.F.Fw.O 9f--(92E2E ) 0Iy(*- +44W=hGI+Ih' 5>((1!00 ,$F=  &hDA   -U3& !0$,,001B1B1F1FG%--11(2C2C2G2GH!0( E +7-4+!&#//@d"S'()*+H0!% 8 882 --3c*D1  II%%%889J9JHUU&&2d6L6L6X%%1&&. 7 A ))5T)# //44#G* 22%$ $ 22>>!C>#99 $ : : > > @54''3&&**(X[[)9=I"-X]]]-M!N))$(:&==?*&&99 93544  LL2TL BG s3N T'A-TB.T T T T9T44T9)rrrorRc0nUb[U5US'UbX6S'UbXFS'UbXVS'URSS[US530URES S 0E[ U5S 9n[ R "U5 g) aUpdate a feedback in the LangSmith API. Args: feedback_id (Union[UUID, str]): The ID of the feedback to update. score (Optional[Union[float, int, bool]]): The score to update the feedback with. value (Optional[Union[float, int, bool, str, dict]]): The value to update the feedback with. correction (Optional[dict]): The correction to update the feedback with. comment (Optional[str]): The comment to update the feedback with. Returns: None NrrrorRr8 /feedback/r}rrr)rrr{rr'r;r)rr}rrrorRfeedback_updaters r?update_feedbackClient.update_feedbacks4+-  '=e'DOG $  ',G $  !,6L )  )0I &,, +}=> ?It}}In6HI_- -  ++H5r>cURSS[US535n[R"S0UR 5D6$)zRead a feedback from the LangSmith API. Args: feedback_id (Union[UUID, str]): The ID of the feedback to read. Returns: Feedback: The feedback. rrr}r=)rr{rbrrrr}rs r? read_feedbackClient.read_feedbacksE,, +}=> ? ""5X]]_55r>)rh feedback_keyrqrc+# UUb [US5OSS.UEnUbX&S'UbX6S'[URSUS95H-upx[R"S 0UD6v UcM"US-U:dM- g g7f) adList the feedback objects on the LangSmith API. Args: run_ids (Optional[Sequence[Union[UUID, str]]]): The IDs of the runs to filter by. feedback_key (Optional[Sequence[str]]): The feedback key(s) to filter by. Examples: 'correctness' The query performs a union of all feedback keys. feedback_source_type (Optional[Sequence[FeedbackSourceType]]): The type of feedback source, such as model or API. limit (Optional[int]): The maximum number of feedback to return. **kwargs (Any): Additional keyword arguments. Yields: The feedback objects. NrU)r`rrsourcerrrr=)rrrrbr) rrhrrqrrrrrs r? list_feedbackClient.list_feedback*s8(-(9S_s    #(5M  +38 $  $ $[ $ @ KA%%11 1 QUe^  sAA2"A2-A2cURSS[US53URS9n[R"U5 g)zDelete a feedback by ID. Args: feedback_id (Union[UUID, str]): The ID of the feedback to delete. Returns: None rrr}rNrrs r?delete_feedbackClient.delete_feedbackUsE,, +}=> ?MM-  ++H5r>)rrorRrc [XRSS9upxXpR:wa[SU35eURSS[ U53[ UUUUUS.5UR S9n [R"U 5 g) aMCreate feedback from a presigned token or URL. Args: token_or_url (Union[str, uuid.UUID]): The token or URL from which to create feedback. score (Optional[Union[float, int, bool]]): The score of the feedback. value (Optional[Union[float, int, bool, str, dict]]): The value of the feedback. correction (Optional[dict]): The correction of the feedback. comment (Optional[str]): The comment of the feedback. metadata (Optional[dict]): Additional metadata for the feedback. Raises: ValueError: If the source API URL is invalid. Returns: None r)rzInvalid source API URL. rz/feedback/tokens/)rrrorRrrN) rrrurr{r'rr;r) rrbrrrorRrr\rrs r?create_feedback_from_token!Client.create_feedback_from_tokenfs8&9 ,,!& " \\ )77GHI I,,  45 6""",& (  MM-  ++H5r>) expirationrpr}cUUUU=(d [[R"55S.nUc[R"SSSS9US'O[ U[ R 5(aUR5US'Ow[ U[ R5(aA[R"URURS-URS-S-S9US'O[S [U535eURS S [U5S 9n[R "U5 [R""S 0UR%5D6$)aCreate a pre-signed URL to send feedback data to. This is useful for giving browser-based clients a way to upload feedback data directly to LangSmith without accessing the API key. Args: run_id (Union[UUID, str]): The ID of the run. feedback_key (str): The key of the feedback to create. expiration (Optional[datetime.datetime | datetime.timedelta]): The expiration time of the pre-signed URL. Either a datetime or a timedelta offset from now. Default to 3 hours. feedback_config (Optional[FeedbackConfig]): If creating a feedback_key for the first time, this defines how the metric should be interpreted, such as a continuous score (w/ optional bounds), or distribution over categorical values. feedback_id (Optional[Union[UUID, str]): The ID of the feedback to create. If not provided, a new feedback will be created. Returns: FeedbackIngestToken: The pre-signed URL for uploading feedback data. )r)rrpr<rrdayshoursminutes expires_in expires_at<Unknown expiration type: r/feedback/tokens)rr=)rtrrrDrbTimeDeltaInputrqr[r, timedeltarsecondsrurrr'r;rFeedbackIngestTokenr)rr)rrrpr}r rs r?create_presigned_feedback_token&Client.create_presigned_feedback_tokens.F(.2TZZ\!2    !+!:!:"D   H$5$5 6 6!+!5!5!7D   H$6$6 7 7!+!:!:__ ((D0#++r1R7"D  8j9I8JKL L,,  T"-  ++H5--@ @@r>)rfeedback_configsc b^^Ub#[U5[U5:wa [S5eU(dS/[U5-nSupVUc[R"SSSS9nO[ U[ R 5(aUR 5nOt[ U[ R5(a>[R"URURS-URS-S-S9nO[S [U535e[[X$5VVs/sH upxUUUUUS .PM snn5mS UU4S jjn /n [R"[TR5S 9n TRR!5V V s/sHupU R#XU 5PM nn n [R$"U5HInUR'5nU R)UVs/sHn[R*"S0UD6PM sn5 MK SSS5 U $s snnfs sn n fs snf!,(df  U $=f)aCreate a pre-signed URL to send feedback data to. This is useful for giving browser-based clients a way to upload feedback data directly to LangSmith without accessing the API key. Args: run_id (Union[UUID, str]): The ID of the run. feedback_keys (Sequence[str]): The key of the feedback to create. expiration (Optional[datetime.datetime | datetime.timedelta]): The expiration time of the pre-signed URL. Either a datetime or a timedelta offset from now. Default to 3 hours. feedback_configs (Optional[Sequence[Optional[FeedbackConfig]]]): If creating a feedback_key for the first time, this defines how the metric should be interpreted, such as a continuous score (w/ optional bounds), or distribution over categorical values. Returns: Sequence[FeedbackIngestToken]: The pre-signed URL for uploading feedback data. NzBThe length of feedback_keys and feedback_configs must be the same.NNrrrrrr)r)rrprrc >TRSUS3T0TRE[U=(d TR0ES.S9n[R "U5 UR 5$)Nrrrr$)rrrrr;rr)rrrr rs r?r4Client.create_presigned_feedback_tokens..req#sn00)+,  -- !7#:dll  1 H  / / 9==? "r>r)rrtrr1rrr=)rzrurbrrqr[r,rrrrr'rr=rrrrrrJr|r)rr) feedback_keysrrrrrrprtokensrrrfutsfutrrr s` @r? create_presigned_feedback_tokens'Client.create_presigned_feedback_tokenss"D  'C ,>#FVBW,WT  $vM(:: !+  #22J  H$5$5 6 6#--/J  H$6$6 7 7#22__ ((D0#++r1R7J 8j9I8JKL L69!6 61L %$0'6",", 6    # #  " "s43G3G/H IX)-(<(<(B(B(D(D$Gg6(D t,::< HPQZ33;d;Q- J Q >RJ I s0H "HH;H H 9 H H H.)rc## [US5Ub [US5OSS.n[URSUS95H-upE[R "S0UD6v UcM"US-U:dM- g g7f) zList the feedback ingest tokens for a run. Args: run_id (Union[UUID, str]): The ID of the run to filter by. limit (Optional[int]): The maximum number of tokens to return. Yields: The feedback ingest tokens. r)NrU)r)rrrrr=)r{rrrrbr)rr)rrrtokens r?list_presigned_feedback_tokens%Client.list_presigned_feedback_tokens?sx vx0(-(9S_s "  $ $%7 $ G HA00959 9 QUe^  sAA,A,'A,rr+rrc## Ub [US5OSUb [US5OSUb [US5OSUS.n[URSUS95H-upg[R "S 0UD6v UcM"US-U:dM- g g7f) aList feedback formulas. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to filter by. session_id (Optional[Union[UUID, str]]): The ID of the session to filter by. limit (Optional[int]): The maximum number of feedback formulas to return. offset (int): The starting offset for pagination. Yields: The feedback formulas. Nrr+rUr/feedback/formulasrrr=)r{rrrrbFeedbackFormula)rrr+rrrrfeedback_formulas r?list_feedback_formulasClient.list_feedback_formulasYs27A6L\2RV7A6L\2RV(-(9S_s " $-  $ $%9& $ I$ A,,@/?@ @ QUe^ $ sA,B2B=BcURSS[US535n[R"U5 [R "S0UR 5D6$)zGet a feedback formula by ID. Args: feedback_formula_id (Union[UUID, str]): The ID of the feedback formula to retrieve. Returns: The requested feedback formula. r/feedback/formulas/feedback_formula_idr=)rr{r;rrbrrrrrs r?get_feedback_formula_by_id!Client.get_feedback_formula_by_idsU,, !(+>@U"V!W X  ++H5)))rr+c UVs/sH:n[U[R5(aUO[R"S 0UD6PM< nn[R"UUUUb [ US5OSUb [ US5OSS9nUR SSS[ URSS 950S 9n [R"U 5 [R"S 0U R5D6$s snf) a.Create a feedback formula. Args: feedback_key (str): The feedback key for the formula. aggregation_type (Literal["sum", "avg"]): The aggregation type to use when combining parts. formula_parts (Sequence[FeedbackFormulaWeightedVariable | dict]): The weighted feedback keys included in the formula. dataset_id (Optional[Union[UUID, str]]): The dataset to scope the formula to. session_id (Optional[Union[UUID, str]]): The session to scope the formula to. Returns: The created feedback formula. Nrr+)raggregation_type formula_partsrr+rrrTrr$r=) rqrbFeedbackFormulaWeightedVariableFeedbackFormulaCreater{rr'rr;rrr) rrrrrr+r typed_partsr3rs r?create_feedback_formulaClient.create_feedback_formulas@& I &$ J JKK ;;CdC D& I 22%-%6@6L\2RV7A6L\2RV  ,,   GLLdL$CD-  ++H5))@U"V!W X GLLdL$CD-  ++H5))@U"V!W X  ++H5r>) queue_idsr1rrc #:# Ub.[U5VVs/sHupV[USUS35PM snnOSUUUb [US5OSS.n[URSUS95H-upX[R "S 0UD6v UcM"US-U:dM- g gs snnf7f) aList the annotation queues on the LangSmith API. Args: queue_ids (Optional[List[Union[UUID, str]]]): The IDs of the queues to filter by. name (Optional[str]): The name of the queue to filter by. name_contains (Optional[str]): The substring that the queue name should contain. limit (Optional[int]): The maximum number of queues to return. Yields: The annotation queues. Nz queue_ids[r6rU)rr1rr/annotation-queuesrrr=)rr{rrrbAnnotationQueue) rrr1rrrrrqueues r?list_annotation_queuesClient.list_annotation_queuess4(AJ)@TU@Tfa#A3a01@TU*(-(9S_s  "  $ $%9& $ I HA,,  QUe^ VsBBABB B)r+queue_idrubric_instructionsc VUUUb [U5O[[R"55US.nURSSUR 5VVs0sH upgUcM Xg_M snnS9n[ R "U5 [R"S0UR5D6$s snnf)aCreate an annotation queue on the LangSmith API. Args: name (str): The name of the annotation queue. description (Optional[str]): The description of the annotation queue. queue_id (Optional[Union[UUID, str]]): The ID of the annotation queue. rubric_instructions (Optional[str]): The rubric instructions for the annotation queue. Returns: AnnotationQueue: The created annotation queue object. )r1r+r<rrrrBr=) rtrrrDrrr;rrbAnnotationQueueWithDetailsr) rr1r+rrr rmrnrs r?create_annotation_queueClient.create_annotation_queue2s0&#+#7#h-S=N#6   ,,  #'::<A<411$!$<A-  ++H544 mmo  Bs  B%B%cS[US53nURSUURS9n[R"U5 [ R "S0UR5D6$)zRead an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue to read. Returns: AnnotationQueue: The annotation queue object. /annotation-queues/rrrr=)r{rrr;rrbrr)rrbase_urlrs r?read_annotation_queueClient.read_annotation_queueYsf)(J)G(HI,, jMM-  ++H544Gx}}GGr>)r+rctURSS[US53UUUS.S9n[R"U5 g)aUpdate an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue to update. name (str): The new name for the annotation queue. description (Optional[str]): The new description for the annotation queue. rubric_instructions (Optional[str]): The new rubric instructions for the annotation queue. Returns: None r8rr)r1r+rrBNr)rrr1r+rrs r?update_annotation_queueClient.update_annotation_queueksL*,, !(8Z"@!A B*':-  ++H5r>cURSS[US53SS0URES9n[R"U5 g)zDelete an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue to delete. Returns: None rrrrrrNr)rrrs r?delete_annotation_queueClient.delete_annotation_queuesP,, !(8Z"@!A B1CT]]C-  ++H5r>cURSS[US5S3[U5VVs/sHup4[[USUS355PM snnS9n[R "U5 gs snnf) aAdd runs to an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. run_ids (List[Union[UUID, str]]): The IDs of the runs to be added to the annotation queue. Returns: None rrrrzrun_ids[r6rBN)rr{rrtr;r)rrrhrrrs r?add_runs_to_annotation_queue#Client.add_runs_to_annotation_queuesu,, !(8Z"@!A GDMgDVWDV&!#hshqcO45DVW-  ++H5Xs%A,c URSS[US5S[US535n[R"U5 g)a Delete a run from an annotation queue with the specified `queue_id` and `run_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. run_id (Union[UUID, str]): The ID of the run to be added to the annotation queue. Returns: None rrrr9r)Nr)rrr)rs r? delete_run_from_annotation_queue'Client.delete_run_from_annotation_queuesI,, !(8Z"@!AQWYaHbGc d  ++H5r>cS[US5S3nURSUSU3URS9n[R"U5 [ R "S0UR5D6$)aGet a run from an annotation queue at the specified index. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. index (int): The index of the run to retrieve. Returns: RunWithAnnotationQueueInfo: The run at the specified index. Raises: LangSmithNotFoundError: If the run is not found at the given index. LangSmithError: For other API-related errors. rrrrrnrr=)r{rrr;rrbRunWithAnnotationQueueInfor)rrr.rrs r?get_run_from_annotation_queue$Client.get_run_from_annotation_queuesq )(J)G(HM,, j% !MM-  ++H544Gx}}GGr>)rr+rlrr<c|U(d [S5eUcURUSS9RnU(d [S5eU=(d [[R "55UUUUU=(d7 [ R R[ RR50S.nUbXhSS'[UR5V V s0sHupX_M sn n 5n URSS S U 0S 9n [R"U 5 U R5n [ R""S 0U D6$s sn n f) a-Create a comparative experiment on the LangSmith API. These experiments compare 2 or more experiment results over a shared dataset. Args: name (str): The name of the comparative experiment. experiments (Sequence[Union[UUID, str]]): The IDs of the experiments to compare. reference_dataset (Optional[Union[UUID, str]]): The ID of the dataset these experiments are compared on. description (Optional[str]): The description of the comparative experiment. created_at (Optional[datetime.datetime]): The creation time of the comparative experiment. metadata (Optional[Dict[str, Any]]): Additional metadata for the comparative experiment. id (Optional[Union[UUID, str]]): The ID of the comparative experiment. Returns: ComparativeExperiment: The created comparative experiment object. z$At least one experiment is required.rrz A reference dataset is required.)r<r1experiment_idsrr+rlrPrPrrz/datasets/comparativerr$r=)rur}rrtrrrDr[r\r]r^r'rrr;rrrbComparativeExperiment)rr1 experimentsrr+rlrr<r rmrnserr response_ds r?create_comparative_experiment$Client.create_comparative_experiments-6CD D  $ $ 1 1&q>!2!"" !?@ @)DJJL))$5&$T(9(9(=(=h>O>O>S>S(T   (0M* %DJJL9LDA14L9:,,  #-  ++H5]]_ //=*==:sD8 cXUR5nUS:H=(d URU:H$)zCheck if the current workspace has the same handle as owner. Args: owner (str): The owner to check against. Returns: bool: True if the current tenant is the owner, False otherwise. -)r tenant_handle)rownersettingss r?_current_tenant_is_ownerClient._current_tenant_is_owners,%%'|>x55>>r>cp[R"SUSUR5RSU35$)NzCannot z% for another tenant. Current tenant: z, Requested tenant: )r;r|rr)ractionrs r?_owner_conflict_errorClient._owner_conflict_error"sH**fX#113AABC!!& )  r>cxURSSUS3X#S.S9nUR5SnU(aUSS$S $) agGet the latest commit hash for a prompt. Args: prompt_owner_and_name (str): The owner and name of the prompt. limit (int, default=1): The maximum number of commits to fetch. Defaults to 1. offset (int, default=0): The number of commits to skip. Defaults to 0. Returns: Optional[str]: The latest commit hash, or None if no commits are found. r /commits/rn)rrrcommitsr commit_hashN)rr)rprompt_owner_and_namerrrrs r?_get_latest_commit_hashClient._get_latest_commit_hash+sX,, -.a 0"5-  --/),,3wqz-(==r>c[R"U5up4nURSSUSU3SU0S9nUR5 UR 5$)aRLike or unlike a prompt. Args: prompt_identifier (str): The identifier of the prompt. like (bool): True to like the prompt, False to unlike it. Returns: A dictionary with the key 'likes' and the count of likes as the value. Raises: requests.exceptions.HTTPError: If the prompt is not found or another error occurs. rz/likes/rnlikerB)r;parse_prompt_identifierrraise_for_statusr)rprompt_identifierrr prompt_namer.rs r?_like_or_unlike_promptClient._like_or_unlike_prompt@sa !) @ @AR SA,, geWAk]364.-  !!#}}r>c [R"U5up#nURU5(dURSUSUSUSS3$UR 5nURSUSUSSSUR 3$)zGet a URL for a prompt. Args: prompt_identifier (str): The identifier of the prompt. Returns: str: The URL for the prompt. z/hub/rnrNz /prompts/z?organizationId=)r;r rrrr<)rr"rr#rrs r?_get_prompt_urlClient._get_prompt_urlWs+3*J*J + 'K,,U33nn%U5';-qRa@QR R%%'~~i }Ak"1o5Fx{{m - r>c:URU5nU(aS$S$)zCheck if a prompt exists. Args: prompt_identifier (str): The identifier of the prompt. Returns: bool: True if the prompt exists, False otherwise. TF) get_prompt)rr"rs r?_prompt_existsClient._prompt_existsns !23t(5(r>c"URUSS9$)zLike a prompt. Args: prompt_identifier (str): The identifier of the prompt. Returns: Dict[str, int]: A dictionary with the key 'likes' and the count of likes as the value. Trr$rr"s r? like_promptClient.like_promptzs**+<4*HHr>c"URUSS9$)zUnlike a prompt. Args: prompt_identifier (str): The identifier of the prompt. Returns: Dict[str, int]: A dictionary with the key 'likes' and the count of likes as the value. Fr/r0r1s r? unlike_promptClient.unlike_prompts**+<5*IIr>rUdesc)rr is_public is_archived sort_fieldsort_directionrdc UUU(aSOUbSOSU(aSOSUUUU(aSOSS.nURSSUS9n [R"S0U R5D6$) aList prompts with pagination. Args: limit (int, default=100): The maximum number of prompts to return. Defaults to 100. offset (int, default=0): The number of prompts to skip. Defaults to 0. is_public (Optional[bool]): Filter prompts by if they are public. is_archived (Optional[bool]): Filter prompts by if they are archived. sort_field (PromptSortField): The field to sort by. Defaults to "updated_at". sort_direction (Literal["desc", "asc"], default="desc"): The order to sort by. Defaults to "desc". query (Optional[str]): Filter prompts by a search query. Returns: ListPromptsResponse: A response object containing the list of prompts. rPNfalse)rrr8r9r:r;rd match_prefixr/repos/rr=)rrbListPromptsResponser) rrrr8r9r:r;rdrrs r? list_promptsClient.list_promptssi:#I4It%06g$,&+F  ,,UIf,M--@ @@r>c[R"U5up#nURSSUSU35n[R"S0UR 5SD6$![R a gf=f)a[Get a specific prompt by its identifier. Args: prompt_identifier (str): The identifier of the prompt. The identifier should be in the format "prompt_name" or "owner/prompt_name". Returns: Optional[Prompt]: The prompt object. Raises: requests.exceptions.HTTPError: If the prompt is not found or another error occurs. rr?rnrepoNr=)r;r rrbPromptrrrr"rr#r.rs r?r+Client.get_promptsu!) @ @AR SA 00'%+8WXH$$?x}}v'>? ?..  s>AA10A1r+readmer&r8cUR5nU(a'UR(d[R"S5e[R"U5upxn UR US9(dUR SU5eUU=(d SU=(d SU=(d /US.n URSSU S9n U R5 [R"S 0U R5S D6$) aCreate a new prompt. Does not attach prompt object, just creates an empty prompt. Args: prompt_identifier (str): The identifier of the prompt. The identifier should be in the formatof owner/name:hash, name:hash, owner/name, or name description (Optional[str]): A description of the prompt. readme (Optional[str]): A readme for the prompt. tags (Optional[Sequence[str]]): A list of tags for the prompt. is_public (bool): Whether the prompt should be public. Returns: Prompt: The created prompt object. Raises: ValueError: If the current tenant is not the owner. HTTPError: If the server request fails. Cannot create a public prompt without first creating a LangChain Hub handle. You can add a handle by creating a public prompt at: https://smith.langchain.com/prompts)rzcreate a promptr) repo_handler+rIr&r8rr?rBrDr=) rrr;r|r rrrr!rbrEr) rr"r+rIr&r8rrr#r.rrs r? create_promptClient.create_prompts8%%' X33--6 !) @ @AR SA,,5,9,,->F F'&,"lJB" < ,,VYT,J!!#  ;8==?6#:;;r>parent_commit_hashcURU5(d[R"S5eSSKJn U"[ U55n[R"U5n[R"U5upxn USU3n US:XdUcURU 5nX6S.n URSS U 3U S 9n U R5S S n URU S U 35$![ a [ S5ef=f)aCreate a commit for an existing prompt. Args: prompt_identifier (str): The identifier of the prompt. object (Any): The LangChain object to commit. parent_commit_hash (Optional[str]): The hash of the parent commit. Defaults to latest commit. Returns: str: The url of the prompt commit. Raises: HTTPError: If the server request fails. ValueError: If the prompt does not exist. z0Prompt does not exist, you must create it first.r)rvThe client.create_commit function requires the langchain-corepackage to run. Install with `pip install langchain-core`rnr ) parent_commitmanifestrrrBcommitrr) r,r;rlangchain_core.loadrrLprep_obj_for_pushrrr rrr()rr"rrPr json_object manifest_dictrr#r.r request_dictrrs r? create_commitClient.create_commits!,""#45511B   1-f56  ;/ ( @ @AR SA#(';- 8  )-?-G!%!=!=>S!T );W ,, i 567l- mmoh/ > ##'<&=Q{m$LMM+ M  s CC/)r+rIr&r8r9cRUR5nU(aUR(d [S5e0nUbX(S'UbX8S'UbXXS'UbXhS'UbXHS'[R"U5upn UR SSU S U 3US 9n U R 5 U R5$) aUpdate a prompt's metadata. To update the content of a prompt, use push_prompt or create_commit instead. Args: prompt_identifier (str): The identifier of the prompt to update. description (Optional[str]): New description for the prompt. readme (Optional[str]): New readme for the prompt. tags (Optional[Sequence[str]]): New list of tags for the prompt. is_public (Optional[bool]): New public status for the prompt. is_archived (Optional[bool]): New archived status for the prompt. Returns: Dict[str, Any]: The updated prompt data as returned by the server. Raises: ValueError: If the prompt_identifier is empty. HTTPError: If the server request fails. rKr+rIr8r9r&r8r?rnrB)rrrur;r rr!r) rr"r+rIr&r8r9rrrr#r.rs r? update_promptClient.update_prompt<s:%%' X336 <>  ""-   #N  )   ""-   L ( @ @AR SA,, wugQ{m44-  !!#}}r>c[R"U5up#nURU5(dURSU5eUR SSUSU35nUR 5 g)zDelete a prompt. Args: prompt_identifier (str): The identifier of the prompt to delete. Returns: bool: True if the prompt was successfully deleted, False otherwise. Raises: ValueError: If the current tenant is not the owner of the prompt. zdelete a promptrr?rnN)r;r rrrr!rFs r? delete_promptClient.delete_promptvsj!) @ @AR SA,,U33,,->F F,,Xq 7VW!!#r> include_modelc [R"U5up4nURSSUSUSUU(aSOS35n[R"S0X4S.UR 5ED6$)zPull a prompt object from the LangSmith API. Args: prompt_identifier (str): The identifier of the prompt. Returns: PromptCommit: The prompt object. Raises: ValueError: If no commits are found for the prompt. rrrnz?include_model=truerrrDr=)r;r rrb PromptCommitr)rr"rdrr#rrs r?pull_prompt_commitClient.pull_prompt_commits"+3*J*J + 'K,, E7!K=+,9(rBD && FhmmoF  r>rrrdc#|# [R"U5upVnUb [SU5OUUUS.nSn X8S'URSSUSUS3US 9n U R 5n U S n U S n U (dgU H.nUbX:a g[ R "S0XVS .UED6v U S - n M0 U[U 5- nX=:agM7f)aList commits for a given prompt. Args: prompt_identifier (str): The identifier of the prompt in the format 'owner/repo_name'. limit (Optional[int]): The maximum number of commits to return. If None, returns all commits. offset (int, default=0): The number of commits to skip before starting to return results. include_model (bool, default=False): Whether to include the model information in the commit data. Yields: A ListedPromptCommit object for each commit. !!! note This method uses pagination to retrieve commits. It will make multiple API calls if necessary to retrieve all commits or up to the specified limit. NrUrjrrrrrnrrrrfrr=)r;r rrrrbListedPromptCommitrz)rr"rrrdrr#r.rrrvalrrits r?list_prompt_commitsClient.list_prompt_commitss0!) @ @AR SA).(9Se_u*  %8 00E7!K=21H --/C NELE$ 33 %AbAQ  c%j F/sB:B<c tSSKJn SSKJn SSKJn SSKJn SSKJ n SSK J nJ n SS KJn UR%XS 9n U "5 U"[&R("U R*55n S S S 5 [-W U5(d+[-X5(a[-U R.U5(a[-X5(aU O8[-X5(a'[-U R.U5(a U R.OS n U c[0R2"S 5eU R4c0U lU R4R7U R8U R:U R<S.5 U(Gay[-X5(Gah[-U R.U5(GaL[?U R@5S:XGa2[-U RBU5(Gd[-U RBU5(a[-U RBRDU5(a[GXR.U RBRD-5n[?UR@5S:XaUUR@SnU "U R.URH"S00U RBRJED6URB5n U $Un U $[-U RBU5(aU R.U RB-n U $U $![a [S5ef=f![a [ R"S 5n GNf=f!,(df  GN=f)aZPull a prompt and return it as a LangChain `PromptTemplate`. This method requires [`langchain-core`](https://pypi.org/project/langchain-core). Args: prompt_identifier: The identifier of the prompt. include_model: Whether to include the model information in the prompt data. Returns: Any: The prompt object in the specified format. r)BaseLanguageModel)r)BaseOutputParser)BasePromptTemplateStructuredPromptRunnableBindingRunnableSequenceztThe client.pull_prompt function requires the langchain-corepackage to run. Install with `pip install langchain-core`)suppress_langchain_beta_warningc3# Sv g7frr=r=r>r?rz;Client.pull_prompt..suppress_langchain_beta_warningssrcNz-Prompt object is not a valid prompt template.) lc_hub_owner lc_hub_repolc_hub_commit_hashrrrr=)&#langchain_core.language_models.baserrlangchain_core.load.loadrlangchain_core.output_parsersrslangchain_core.promptsrt!langchain_core.prompts.structuredrvlangchain_core.runnables.baserxryrLlangchain_core._apirz contextlibcontextmanagerrhrrrTrqfirstr;rrrrrDrrzstepslastboundrbindr)rr"rdrrrrsrtrvrxryrz prompt_objectrprompt_templateseq rebound_llms r? pull_promptClient.pull_prompts  M 6 F A J W   K// 0 - .4::m&<&<=>F/ v1 2 2&336<<);<<f99"&;;"6<<1CDDLL &--C''/+-(  $ $ + +$1$7$7#0#5#5*7*C*C  6446<<)9::FLL!Q&z&++GW/X/X&++77J !!#4==+\\FKKQ&"%))A,K- #((B+Afkk.@.@+ABF !F FKK):;;+1<<&+++E  _ M     & & '  / .s(&K(L'L((K> L%$L%( L7)rrPr8r+rIr&cURU5(a-[SXEXg455(aURUUUUUS9 OURUUbUOSUUUS9 UcUR US9$UR UUUS9nU$)aPush a prompt to the LangSmith API. Can be used to update prompt metadata or prompt content. If the prompt does not exist, it will be created. If the prompt exists, it will be updated. Args: prompt_identifier (str): The identifier of the prompt. object (Optional[Any]): The LangChain object to push. parent_commit_hash (str): The parent commit hash. Defaults to "latest". is_public (Optional[bool]): Whether the prompt should be public. If None (default), the current visibility status is maintained for existing prompts. For new prompts, None defaults to private. Set to True to make public, or False to make private. description (Optional[str]): A description of the prompt. Defaults to an empty string. readme (Optional[str]): A readme for the prompt. Defaults to an empty string. tags (Optional[Sequence[str]]): A list of tags for the prompt. Defaults to an empty list. Returns: str: The URL of the prompt. c3(# UHoSLv M g7frr=)rparams r?r%Client.push_prompt..ns/UeT!/UsrHF)r8r+rIr&)r"rO)r,anyr^rMr(r[) rr"rrPr8r+rIr&rs r? push_promptClient.push_promptGsL   0 1 109/U""% +!' #   !'0'<)%'   >'':K'L L   1!   r>cSUlg)z2Manually trigger cleanup of the background thread.TN)r$rs r?cleanupClient.cleanups #r>c  grr=rrJr evaluatorssummary_evaluatorsrexperiment_prefixr+rnum_repetitionsblocking experimentupload_resultsrs r?evaluateClient.evaluates" r>c  grr=rs r?rrs"(+r>c <SSKJn U"U4UUUUUUUU UU U U U S. UD6$)a6Evaluate a target system on a given dataset. Args: target (Union[TARGET_T, Runnable, EXPERIMENT_T, Tuple[EXPERIMENT_T, EXPERIMENT_T]]): The target system or experiment(s) to evaluate. Can be a function that takes a `dict` and returns a `dict`, a langchain `Runnable`, an existing experiment ID, or a two-tuple of experiment IDs. data (DATA_T): The dataset to evaluate on. Can be a dataset name, a list of examples, or a generator of examples. evaluators (Optional[Union[Sequence[EVALUATOR_T], Sequence[COMPARATIVE_EVALUATOR_T]]]): A list of evaluators to run on each example. The evaluator signature depends on the target type. Default to None. summary_evaluators (Optional[Sequence[SUMMARY_EVALUATOR_T]]): A list of summary evaluators to run on the entire dataset. Should not be specified if comparing two existing experiments. metadata (Optional[dict]): Metadata to attach to the experiment. experiment_prefix (Optional[str]): A prefix to provide for your experiment name. description (Optional[str]): A free-form text description for the experiment. max_concurrency (Optional[int], default=0): The maximum number of concurrent evaluations to run. If `None` then no limit is set. If `0` then no concurrency. blocking (bool, default=True): Whether to block until the evaluation is complete. num_repetitions (int, default=1): The number of times to run the evaluation. Each item in the dataset will be run and evaluated this many times. Defaults to 1. experiment (Optional[EXPERIMENT_T]): An existing experiment to extend. If provided, `experiment_prefix` is ignored. For advanced usage only. Should not be specified if target is an existing experiment or two-tuple fo experiments. upload_results (bool, default=True): Whether to upload the results to LangSmith. error_handling (str, default="log"): How to handle individual run errors. `'log'` will trace the runs with the error message as part of the experiment, `'ignore'` will not count the run as part of the experiment at all. **kwargs (Any): Additional keyword arguments to pass to the evaluator. Returns: ExperimentResults: If target is a function, Runnable, or existing experiment. ComparativeExperimentResults: If target is a two-tuple of existing experiments. Examples: Prepare the dataset: ```python from langsmith import Client client = Client() dataset = client.clone_public_dataset( "https://smith.langchain.com/public/419dcab2-1d66-4b94-8901-0357ead390df/d" ) dataset_name = "Evaluate Examples" ``` Basic usage: ```python def accuracy(outputs: dict, reference_outputs: dict) -> dict: # Row-level evaluator for accuracy. pred = outputs["response"] expected = reference_outputs["answer"] return {"score": expected.lower() == pred.lower()} ``` ```python def precision(outputs: list[dict], reference_outputs: list[dict]) -> dict: # Experiment-level evaluator for precision. # TP / (TP + FP) predictions = [out["response"].lower() for out in outputs] expected = [ref["answer"].lower() for ref in reference_outputs] # yes and no are the only possible answers tp = sum([p == e for p, e in zip(predictions, expected) if p == "yes"]) fp = sum([p == "yes" and e == "no" for p, e in zip(predictions, expected)]) return {"score": tp / (tp + fp)} def predict(inputs: dict) -> dict: # This can be any function or just an API call to your app. return {"response": "Yes"} results = client.evaluate( predict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment", description="Evaluating the accuracy of a simple prediction model.", metadata={ "my-prompt-version": "abcd-1234", }, ) ``` Evaluating over only a subset of the examples ```python experiment_name = results.experiment_name examples = client.list_examples(dataset_name=dataset_name, limit=5) results = client.evaluate( predict, data=examples, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment", description="Just testing a subset synchronously.", ) ``` Streaming each prediction to more easily + eagerly debug. ```python results = client.evaluate( predict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], description="I don't even have to block!", blocking=False, ) for i, result in enumerate(results): # doctest: +ELLIPSIS pass ``` View the evaluation results for experiment:... Evaluating a LangChain object: ```python from langchain_core.runnables import chain as as_runnable @as_runnable def nested_predict(inputs): return {"response": "Yes"} @as_runnable def lc_predict(inputs): return nested_predict.invoke(inputs) results = client.evaluate( lc_predict, data=dataset_name, evaluators=[accuracy], description="This time we're evaluating a LangChain object.", summary_evaluators=[precision], ) ``` Comparative evaluation: ```python results = client.evaluate( # The target is a tuple of the experiment IDs to compare target=( "12345678-1234-1234-1234-123456789012", "98765432-1234-1234-1234-123456789012", ), evaluators=[accuracy], summary_evaluators=[precision], ) ``` Evaluate an existing experiment: ```python results = client.evaluate( # The target is the ID of the experiment we are evaluating target="12345678-1234-1234-1234-123456789012", evaluators=[accuracy], summary_evaluators=[precision], ) ``` !!! version-added "Added in `langsmith` 0.2.0" r)r rrrrrr+rrclientrrrerror_handling)langsmith.evaluation._runnerr)rrJrrrrrr+rrrrrrr evaluate_s r?rrsQ\ G  !1/#++!))   r>cX# SSKJn U"U4UUUUUUUU UU U U U S. UD6IShvN $N7f)aYEvaluate an async target system on a given dataset. Args: target (Union[ATARGET_T, AsyncIterable[dict], Runnable, str, uuid.UUID, TracerSession]): The target system or experiment(s) to evaluate. Can be an async function that takes a `dict` and returns a `dict`, a langchain `Runnable`, an existing experiment ID, or a two-tuple of experiment IDs. data (Union[DATA_T, AsyncIterable[Example]]): The dataset to evaluate on. Can be a dataset name, a list of examples, an async generator of examples, or an async iterable of examples. evaluators (Optional[Sequence[EVALUATOR_T]]): A list of evaluators to run on each example. summary_evaluators (Optional[Sequence[SUMMARY_EVALUATOR_T]]): A list of summary evaluators to run on the entire dataset. metadata (Optional[dict]): Metadata to attach to the experiment. experiment_prefix (Optional[str]): A prefix to provide for your experiment name. description (Optional[str]): A description of the experiment. max_concurrency (Optional[int], default=0): The maximum number of concurrent evaluations to run. If `None` then no limit is set. If `0` then no concurrency. num_repetitions (int, default=1): The number of times to run the evaluation. Each item in the dataset will be run and evaluated this many times. Defaults to 1. blocking (bool, default=True): Whether to block until the evaluation is complete. experiment (Optional[TracerSession]): An existing experiment to extend. If provided, `experiment_prefix` is ignored. For advanced usage only. upload_results (bool, default=True): Whether to upload the results to LangSmith. error_handling (str, default="log"): How to handle individual run errors. `'log'` will trace the runs with the error message as part of the experiment, `'ignore'` will not count the run as part of the experiment at all. **kwargs (Any): Additional keyword arguments to pass to the evaluator. Returns: An async iterator over the experiment results. Environment: - `LANGSMITH_TEST_CACHE`: If set, API calls will be cached to disk to save time and cost during testing. Recommended to commit the cache files to your repository for faster CI/CD runs. Requires the `'langsmith[vcr]'` package to be installed. Examples: Prepare the dataset: ```python import asyncio from langsmith import Client client = Client() dataset = client.clone_public_dataset( "https://smith.langchain.com/public/419dcab2-1d66-4b94-8901-0357ead390df/d" ) dataset_name = "Evaluate Examples" ``` Basic usage: ```python def accuracy(outputs: dict, reference_outputs: dict) -> dict: # Row-level evaluator for accuracy. pred = outputs["resposen"] expected = reference_outputs["answer"] return {"score": expected.lower() == pred.lower()} def precision(outputs: list[dict], reference_outputs: list[dict]) -> dict: # Experiment-level evaluator for precision. # TP / (TP + FP) predictions = [out["response"].lower() for out in outputs] expected = [ref["answer"].lower() for ref in reference_outputs] # yes and no are the only possible answers tp = sum([p == e for p, e in zip(predictions, expected) if p == "yes"]) fp = sum([p == "yes" and e == "no" for p, e in zip(predictions, expected)]) return {"score": tp / (tp + fp)} async def apredict(inputs: dict) -> dict: # This can be any async function or just an API call to your app. await asyncio.sleep(0.1) return {"response": "Yes"} results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment", description="Evaluate the accuracy of the model asynchronously.", metadata={ "my-prompt-version": "abcd-1234", }, ) ) ``` Evaluating over only a subset of the examples using an async generator: ```python async def example_generator(): examples = client.list_examples(dataset_name=dataset_name, limit=5) for example in examples: yield example results = asyncio.run( client.aevaluate( apredict, data=example_generator(), evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Subset Experiment", description="Evaluate a subset of examples asynchronously.", ) ) ``` Streaming each prediction to more easily + eagerly debug. ```python results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Streaming Experiment", description="Streaming predictions for debugging.", blocking=False, ) ) async def aenumerate(iterable): async for elem in iterable: print(elem) asyncio.run(aenumerate(results)) ``` Running without concurrency: ```python results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment Without Concurrency", description="This was run without concurrency.", max_concurrency=0, ) ) ``` Using Async evaluators: ```python async def helpfulness(outputs: dict) -> dict: # Row-level evaluator for helpfulness. await asyncio.sleep(5) # Replace with your LLM API call return {"score": outputs["output"] == "Yes"} results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[helpfulness], summary_evaluators=[precision], experiment_prefix="My Helpful Experiment", description="Applying async evaluators example.", ) ) ``` Evaluate an existing experiment: ```python results = asyncio.run( client.aevaluate( # The target is the ID of the experiment we are evaluating target="419dcab2-1d66-4b94-8901-0357ead390df", evaluators=[accuracy, helpfulness], summary_evaluators=[precision], ) ) ``` !!! version-added "Added in `langsmith` 0.2.0" r) aevaluaterN)langsmith.evaluation._arunnerr)rrJrrrrrr+rrrrrrr aevaluate_s r?rClient.aevaluate s\N J  !1/#++!))     s !*(*c## SnSnU(aXh- OSn U (a [SU 5OSn U/UU UUUS.n URSSUS3S[U 50S 9n U R5n U (dgU Vs/sHn[R "S 0UD6PM nnUv U[ U 5- n[ U 5U :d U(aX:agU[ U 5- nMs snf7f) aPaginate through examples with runs and yield batches. Args: dataset_id: Dataset UUID to fetch examples with runs session_id: Session UUID to filter runs by, same as project_id preview: Whether to return preview data only comparative_experiment_id: Optional comparative experiment UUID filters: Optional filters to apply limit: Maximum total number of results to return Yields: Batches of run results as lists of ExampleWithRuns instances rNrU) session_idsrrpreviewr~filtersrrrrr$r=)rrr'rrbr6rz)rrr+rr~rrr results_count remaining batch_limitr rrrJexamples_batchs r?_paginate_examples_with_runs#Client._paginate_examples_with_runs!s, 38.dI1:#c9-K!+| $"-F" D00ZL. & D(9:1H MMOERWWQVvj88B6BQVNW SZ 'M5zK'Em6L c%j F?0XsA.C0 CACc8^^^^^^ TRXSS9m T (d[SUS35eUUUUU U4SjnT RT RT RT R T R T RT RT RT RT RT RT RT RT RS.n0T R =(d 0ET R"=(d 0En [$R&"U UU"5S9$)aGet results for an experiment, including experiment session aggregated stats and experiment runs for each dataset example. Experiment results may not be available immediately after the experiment is created. Args: name: The experiment name. project_id: Experiment's tracing project id, also called session_id, can be found in the url of the LS experiment page preview: Whether to return lightweight preview data only. When True, fetches inputs_preview/outputs_preview summaries instead of full inputs/outputs from S3 storage. Faster and less bandwidth. comparative_experiment_id: Optional comparative experiment UUID for pairwise comparison experiment results. filters: Optional filters to apply to results limit: Maximum number of results to return Returns: ExperimentResults with: - feedback_stats: Combined feedback statistics including session-level feedback - run_stats: Aggregated run statistics (latency, tokens, cost, etc.) - examples_with_runs: Iterator of ExampleWithRuns Raises: ValueError: If project not found for the given session_id Example: ```python client = Client() results = client.get_experiment_results( project_id="037ae90f-f297-4926-b93c-37d8abf6899f", ) for example_with_runs in results["examples_with_runs"]: print(example_with_runs.dict()) # Access aggregated experiment statistics print(f"Total runs: {results['run_stats']['run_count']}") print(f"Total cost: {results['run_stats']['total_cost']}") print(f"P50 latency: {results['run_stats']['latency_p50']}") # Access feedback statistics print(f"Feedback stats: {results['feedback_stats']}") ``` T)r~rcrz&No experiment found with project_id: 'rc 3># TRTRTRTTTTS9H nUShvN M gN 7f)z2Yield examples with corresponding experiment runs.)rr+rr~rrN)rrr<)rr~rrrrrs r? _get_examples_with_runs_iteratorGClient.get_experiment_results.._get_examples_with_runs_iterator"sL::"77"::*C ;!  !s1A> A) run_count latency_p50 latency_p99rvrsrnlast_run_start_time run_facetsrurrrmfirst_token_p50first_token_p99 error_rate)ro run_statsexamples_with_runs)r}rurrrrvrsrnrrrurrrmrrrrosession_feedback_statsrbri) rr1rcrr~rrrrrors ` ```` @r?get_experiment_resultsClient.get_experiment_results!sd##D$ Ej\QRST T ! !!**".."..#00$22!(!:!:#*#>#>!,,!,,"..&66&66&66!,,4  %%+ --3 ++)?A  r>) instructionsr1modelopenai_api_keyanthropic_api_keycURUUUS9nURX5nUSUS.SUS.nURSSURS3US 9n [R "U 5 U R 5n [R"S 0U DURUR5URS .D6n [S U R35 U $) ajGenerate Insights over your agent chat histories. !!! note - Only available to Plus and higher tier LangSmith users. - Insights Agent uses user's model API key. The cost of the report grows linearly with the number of chat histories you upload and the size of each history. For more see [insights](https://docs.langchain.com/langsmith/insights). - This method will upload your chat histories as traces to LangSmith. - If you pass in a model API key this will be set as a workspace secret meaning it will be usedin for evaluators and the playground. Args: chat_histories: A list of chat histories. Each chat history should be a list of messages. We recommend formatting these as OpenAI messages with a "role" and "content" key. Max length 1000 items. instructions: Instructions for the Insights agent. Should focus on what your agent does and what types of insights you want to generate. name: Name for the generated Insights report. model: Whether to use OpenAI or Anthropic models. This will impact the cost of generating the Insights Report. openai_api_key: OpenAI API key to use. Only needed if you have not already stored this in LangSmith as a workspace secret. anthropic_api_key: Anthropic API key to use. Only needed if you have not already stored this in LangSmith as a workspace secret. Example: ```python import os from langsmith import Client client = client() chat_histories = [ [ {"role": "user", "content": "how are you"}, {"role": "assistant", "content": "good!"}, ], [ {"role": "user", "content": "do you like art"}, {"role": "assistant", "content": "only Tarkovsky"}, ], ] report = client.generate_insights( chat_histories=chat_histories, name="Conversation Topics", instructions="What are the high-level topics of conversations users are having with the assistant?", openai_api_key=os.environ["OPENAI_API_KEY"], ) # client.poll_insights(report=report) ``` rrrzxThe run.outputs.messages field contains a chat history between the user and the agent. This is all the context you need.)z%How are your agent traces structured?z.What would you like to learn about your agent?r)r1 user_context last_n_hoursrrrz /insightsrBrcrhost_urlzThe Insights Agent is running! This can take up to 30 minutes to complete. Once the report is completed, you'll be able to see results here: r=) _ensure_insights_api_key_ingest_insights_runsrr<r;rrrbInsightsReportrrprintr) rchat_historiesrr1rrrrrrrwreports r?generate_insightsClient.generate_insights."sD--)/.  ,,^B:tBN ,, j I6V-  ++H5mmo**  zz))+^^    R{{m   r>ri)rr<rcraterverbosec U(aU(dU(d [S5eU(dU(aU(a [S5eU(aURnURn[SXT-5n[ U5HnUR SSUSU35n [ R"U 5 U R5n U SS:XaN[R"S0U DUUR5URS .D6n [S U R5 U s $U SS :Xa[S U S 35eU(a[S X-35 [R "U5 M [#S5e)zPoll the status of an Insights report. Args: report: THe InsightsReport. id: The Insights report ID. Should only specify if 'report' is not specified. project_id: The Tracing project ID. Should only specify if 'report' is not specified. z1Must specify ('id' and 'project_id') or 'report'.z@Must specify exactly one of ('id' and 'project_id') or 'report'.rrrz /insights/rtsuccessrz1Insights report completed! View the results at %srzFailed to generate insights: zPolling time: zInsights still pendingr=)rur<rcrrrr;rrrbrrrrrrrr TimeoutError) rrr<rcrrr max_triesrr resp_jsonjobs r? poll_insightsClient.poll_insights"sP$ vPQ QJFR B**J7?+ y!A00JH  / / 9 I"i/ //)"113!^^  GHH 8$/ #@7AS@T!UVVqxj12 JJt -".344r>rcURSS5n[R"U5 UR5Vs1sHoUR S5iM nn[ 5nUS;aUR [5 US;aUR [5 URU5=n(a[U;aS$S$US:Xa Un [n OGUS:Xa Un [n O8U(dU(aU=(d Un U(a[O[n O [S5eURS SXS ./S 9n[R"U5 U [:XaS$S$s snf) Nrz/workspaces/current/secretsr)Nopenai)N anthropicrrz1Must specify openai_api_key or anthropic_api_key.r)rrrB) rr;rrrrZrz_OPENAI_API_KEY_ANTHROPIC_API_KEY intersectionru) rrrrrsworkspace_keys target_keys existing_keysrapi_vars r?rClient._ensure_insights_api_key"s6,,U4QR++H508 @1%%,@e $ $ OOO , ' ' OO. /*77 D D= D.-?8 P[ P h $G%G k !'G(G 0$9(9G)7o=OGPQ Q,,  )!45-  ++H5"o5xF;F5AsE c[U5S:a[R"S5 USSn[RR [R R 5nURU=(d0 S[RR 5RS5-5n[[U55Vs/sH!n[[R"55PM# nn[Xa5VVs/sHTupxSUSS0SU0UUURS5[U53U[R"SS9- US URS S . PMV n nnUR!U S 9 UR#5 U$s snfs snnf) NrIzDCan only generate insights over 1000 data. Truncating to first 1000.z insights z%Y-%m-%d %H:%M:%Srurz%Y%m%dT%H%M%S%fZ)rrrC) rNrOr<rwrrQr%rSr+r1)r)rzrrr[r\r]r^rstrftimerrtrrrDrrr<rrF) rrr1r\rr.rhr)rrs r?rClient._ingest_insights_runs"sh t9t  MMV ;D##H$5$5$9$9:%%  Uh//335>>?RSS /4CI.>?.>3tzz|$.>?!/ 0 &q!u-&?"#&<<0B#C"DS[M R!H$6$6q$AA#%jj 0   d+ %@ s (E78AE<)'rrr'rr(rrrrr"r$r.r0rr,rr%r)r!r*rr r-r#rrr/rr rrrr&r+rrrrrr)2rr1rr1rzOptional[Retry]rz%Optional[Union[int, tuple[int, int]]]r3r1rzOptional[requests.Session]r4rr5z Optional[Callable[[dict], dict]]r6-Optional[Union[Callable[[dict], dict], bool]]r7rr8rr9rr:z4Optional[Callable[[Sequence[dict]], Sequence[dict]]]r; Optional[int]r<Optional[float]r=z/Optional[Union[dict, ls_schemas.LangSmithInfo]]r>Optional[dict[str, str]]r?zOptional[TracerProvider]r@Optional[bool]rrrAr1rBrrCrrDz%Optional[Callable[[Exception], None]]rr)rrt)rrrr)rr2)rrtrrrr)rr1)rr1rr)rzls_schemas.LangSmithInfo)rzls_schemas.LangSmithSettings)rrrr1)rz0Literal['GET', 'POST', 'PUT', 'PATCH', 'DELETE']rrtrzOptional[Mapping]rrr'Optional[Sequence[type[BaseException]]]rrrz1Optional[Callable[[requests.Response, int], Any]]rrtrrrzrequests.Response)rxrtrOptional[dict]rIterator[dict]) rxrtr rr!zLiteral['GET', 'POST']r"rtrr)r7 pd.DataFramer1rtr/ Sequence[str]r0rr+r1r,Optional[ls_schemas.DataType]rls_schemas.Dataset)r8z"Union[str, tuple[str, io.BytesIO]]r/rr0rr1r1r+r1r,rrr)FF)r`z3Union[ls_schemas.Run, dict, ls_schemas.RunLikeDict]rrrrrr)rzSequence[dict]rrrr)rzIterable[dict]rurrz list[dict])r1rtrNdict[str, Any]rS RUN_TYPE_Tr~r1rr1rrrr1rr1rrrr)rarrr1rr1rr)rarrr1rr1)rNr)rOr)rrrr)rr)rzlist[SerializedRunOperation]rr1rr1rrr)rGOptional[Sequence[Union[ls_schemas.Run, ls_schemas.RunLikeDict, dict]]]rr rrrr)r bytesrrtrr1rr1)rz@list[Union[SerializedRunOperation, SerializedFeedbackOperation]]rr1rr1rr) rr rr rrrrrr)rr)rrrr1rr1)r!z io.BytesIOr"zOptional[tuple[int, int]]rr)$r)ID_TYPEr1r1rSzOptional[RUN_TYPE_T]rQOptional[datetime.datetime]r%r rr1rNrrOrrzOptional[Sequence[dict]]rPrr&Optional[list[str]]r Optional[ls_schemas.Attachments]rrr'str | uuid.UUID | Nonerr1rr1rrrr)r5rrr1rr1)r5rrr1rr1rr)r)rrrr)r`ls_schemas.Runrrr)r)r r`rrr)$rcz+Optional[Union[ID_TYPE, Sequence[ID_TYPE]]]r~z#Optional[Union[str, Sequence[str]]]rSr1rwOptional[ID_TYPE]r'rrdr1rer1rfr1rgr1rIrr*rrQr rrrhOptional[Sequence[ID_TYPE]]riOptional[Sequence[str]]rrrrrIterator[ls_schemas.Run])"r<Optional[list[ID_TYPE]]rCrryrrSr1rrrrrrrQr1r%r1rrrdr1rer1rfr1rgr1rIrrr1rr)r`zls_schemas.RunBaser~r1rcrrrt)r)r rrrrt)r)r rr)r)r rr1)r)r rr)rUnion[ID_TYPE, str]r)rrr)rrrhrrr)rrrr1rzls_schemas.DatasetShareSchema)rr rr)rrtrr)rrtrrrrrIterator[ls_schemas.Example]) rrtrrr1r1rr1rrr(Iterator[ls_schemas.TracerSessionResult])r~rtr+r1rrrrrrrrrls_schemas.TracerSession)rcr r1r1r+r1rrrrr%r rr)rOptional[uuid.UUID])r uuid.UUID)rcr1r~r1rrrzls_schemas.TracerSessionResult)r~rtrcr1rr)rcrr~r1rr) NNNNNNNNNN)rrr1r1rr1rrr r1rrrrrr1rrrOptional[dict[str, Any]]rr)r~r1rcr1rr)rrtr+r1r,ls_schemas.DataTyper&rr'rr(z0Optional[list[ls_schemas.DatasetTransformation]]rrrr)rr1rrrr)rr1rrrr) rrrr1r9Union[str, datetime.datetime]r:rrzls_schemas.DatasetDiffInfo)rrrr1rr)rErr,r1rr1rFr1rrrrrzIterator[ls_schemas.Dataset])rrrr1rr) rrrr1rOzdatetime.datetimerPrtrr) rrrr1rTr1rrrz#Iterator[ls_schemas.DatasetVersion]) rrrr1rOr rPr1rzls_schemas.DatasetVersion)rbrtr\r1rr1rr)rr rr)NNNN) rrtrnr1rrrr1rlr rls_schemas.Example) ruz:list[Union[Mapping[str, Any], ls_schemas.BaseMessageLike]]rvz>Optional[Union[Mapping[str, Any], ls_schemas.BaseMessageLike]]rrrr1rlr rr r) r`rrrrr1rlr rr )rzwUnion[list[ls_schemas.ExampleCreate] | list[ls_schemas.ExampleUpsertWithAttachments] | list[ls_schemas.ExampleUpdate],]rrrrrz/tuple[Any, bytes, dict[str, io.BufferedReader]])rr rz(Optional[list[ls_schemas.ExampleUpdate]]rrr!ls_schemas.UpsertExamplesResponse)rr rz(Optional[list[ls_schemas.ExampleCreate]]rrrr!)rzls_schemas.ExampleCreaterr)r)rzlist[ls_schemas.ExampleCreate]rBrrz$list[list[ls_schemas.ExampleCreate]])rz7Optional[list[ls_schemas.ExampleUpsertWithAttachments]]rrrr!)rr1rrrz3Optional[Sequence[ls_schemas.ExampleCreate | dict]]rrrz!Annotated[int, Field(ge=1, le=3)]rrrz2ls_schemas.UpsertExamplesResponse | dict[str, Any]) NNNNNNNNNFNN)rNOptional[Mapping[str, Any]]rrrr1rlr rOr"rr"ryOptional[str | list[str]]rrrrrrrrrrrr )rr rOr rr )NNNNNT)rrrr1rrrOz'Optional[Union[datetime.datetime, str]]rrrrrrrrrrrer1rrrrrr)rr rPrtrrrr)rr rrrr) rNrrrrr rer1rrrzlist[ls_schemas.ExampleSearch])rr rNrrOr"rrryr#rrrz*Optional[ls_schemas.AttachmentsOperations]rrrr) r str | NonerzID_TYPE | Nonerz3Optional[Sequence[ls_schemas.ExampleUpdate | dict]]rrrrrr)rr rr)rSequence[ID_TYPE]r3rrr)rrrr1rOz'Optional[Union[str, datetime.datetime]]rz list[str]) rrrr1rArtr list[ID_TYPE]ryrrr)r`9Union[ls_schemas.Run, ls_schemas.RunBase, str, uuid.UUID]r`rrr)rz5Union[ls_schemas.Example, str, uuid.UUID, dict, None]r`rrzOptional[ls_schemas.Example])rJUnion[ls_evaluator.EvaluationResult, ls_evaluator.EvaluationResults, dict]rNr1r#list[ls_evaluator.EvaluationResult]) r`r'r^ls_evaluator.RunEvaluatorr`rrx9Optional[Union[ls_schemas.Example, str, dict, uuid.UUID]]r`rrr[) rgr(r`zOptional[ls_schemas.Run]r`rrcrrizOptional[cf.ThreadPoolExecutor]rr)) r`z%Union[ls_schemas.Run, str, uuid.UUID]r^r*r`rrxr+r`rrr[)Nunnamed)(r)rrrtrUnion[float, int, bool, None]rzUnion[str, dict, None]rwrroUnion[dict, None]rRUnion[str, None]r`rrqz)Union[ls_schemas.FeedbackSourceType, str]rrr}rrp#Optional[ls_schemas.FeedbackConfig]rrrcrr~rrrrPrrrrrrls_schemas.Feedback) r}r rr-r(Union[float, int, bool, str, dict, None]ror.rRr/rr)r}r rr1) rhrrrrqz1Optional[Sequence[ls_schemas.FeedbackSourceType]]rrrrrzIterator[ls_schemas.Feedback])r}r rr)rbUnion[str, uuid.UUID]rr-rr2ror.rRr/rrrr) r)r rrtr0Optional[datetime.datetime | datetime.timedelta]rpr0r}rrzls_schemas.FeedbackIngestToken) r)r rrrr4rz7Optional[Sequence[Optional[ls_schemas.FeedbackConfig]]]rz(Sequence[ls_schemas.FeedbackIngestToken])r)r rrrz(Iterator[ls_schemas.FeedbackIngestToken]) rrr+rrrrrrz$Iterator[ls_schemas.FeedbackFormula])rr rls_schemas.FeedbackFormula) rrtrLiteral['sum', 'avg']rASequence[Union[ls_schemas.FeedbackFormulaWeightedVariable, dict]]rrr+rrr5) rr rrtrr6rr7rr5)rr rr) rrr1r1rr1rrrz$Iterator[ls_schemas.AnnotationQueue]) r1rtr+r1rrrr1rz%ls_schemas.AnnotationQueueWithDetails)rr rzls_schemas.AnnotationQueue) rr r1rtr+r1rr1rr)rr rr)rr rhr&rr)rr r)r rr)rr r.rrz%ls_schemas.RunWithAnnotationQueueInfo)r1rtrr%rrr+r1rlr rrr<rrz ls_schemas.ComparativeExperiment)rrtrr)rrtrrtrzls_utils.LangSmithUserError)rr)rrtrrrrrr1)r"rtrrrdict[str, int])r"rtrrt)r"rtrr)r"rtrr8)rrrrr8rr9rr:zls_schemas.PromptSortFieldr;zLiteral['desc', 'asc']rdr1rzls_schemas.ListPromptsResponse)r"rtrzOptional[ls_schemas.Prompt]) r"rtr+r1rIr1r&rr8rrzls_schemas.Prompt)r"rtrrrPr1rrt)r"rtr+r1rIr1r&rr8rr9rrr)r"rtrr)r"rtrdrrzls_schemas.PromptCommit) r"rtrrrrrdrrz'Iterator[ls_schemas.ListedPromptCommit])r"rtrdrrr)r"rtrz Optional[Any]rPrtr8rr+r1rIr1r&rrrt) NNNNNNrrTNT)rOptional[DATA_T]rzOptional[Sequence[EVALUATOR_T]]r'Optional[Sequence[SUMMARY_EVALUATOR_T]]rrrr1r+r1rrrrrrrOptional[EXPERIMENT_T]rrrJz'Union[TARGET_T, Runnable, EXPERIMENT_T]rrrri)rr9rz+Optional[Sequence[COMPARATIVE_EVALUATOR_T]]rr:rrrr1r+r1rrrrrrrr;rrrJz(Union[tuple[EXPERIMENT_T, EXPERIMENT_T]]rrrrh) NNNNNNrrTNTr)rr9rzIOptional[Union[Sequence[EVALUATOR_T], Sequence[COMPARATIVE_EVALUATOR_T]]]rr:rrrr1r+r1rrrrrrrr;rrrLiteral['log', 'ignore']rJzJUnion[TARGET_T, Runnable, EXPERIMENT_T, tuple[EXPERIMENT_T, EXPERIMENT_T]]rrrz6Union[ExperimentResults, ComparativeExperimentResults])rzNUnion[DATA_T, AsyncIterable[schemas.Example], Iterable[schemas.Example], None]rz4Optional[Sequence[Union[EVALUATOR_T, AEVALUATOR_T]]]rr:rrrr1r+r1rrrrrrrz6Optional[Union[schemas.TracerSession, str, uuid.UUID]]rrrr<rJzVUnion[ATARGET_T, AsyncIterable[dict], Runnable, str, uuid.UUID, schemas.TracerSession]rrrra)FNNN)rr r+rrrr~rr!dict[uuid.UUID, list[str]] | NonerrrzIterator[list[ExampleWithRuns]])NNFNNN)r1r1rcrrrr~rrr=rrrzls_schemas.ExperimentResults)rzlist[list[dict]]rrtr1r$r%Literal['openai', 'anthropic'] | Nonerr$rr$rls_schemas.InsightsReport)rz ls_schemas.InsightsReport | Noner<rrcrrrrrrrrr?)rr$rr$rr>rzLiteral['openai', 'anthropic'])rrr1r$)rSrTrUrVrW __slots____annotations__rrrrpropertyrrrrrsetterrAr=rrrrr)rbr_r`r9r6rbrorrr|rrrrXrYrZrrrrrrr rr#r0r-r4rCrFrXrarOrrrrrrrrrrrrrrrrrGrr;xor_argsr}rrr!r$r.r2rr>rCrHrKrRrWrZrerqrorzrrrrrrrrrrarrrmrrr"r rrrr.r1r9r>rCrHrLr^rdrer{rurrlrrrrrrrrrrrrrrrrrrrrrr rrrr$r(r,r2r5PromptSortField updated_atrAr+rMr[r^rarhrorrrrrrrrDEFAULT_INSTRUCTIONSrrrrrXr=r>r?r r s'8*IX##!!"&E>"&(,<@!%.2#'7;EIFJGK). -159@D-19='+15&*.2,0HL9E>E> E> & E> : E>E>,E>!E>5E>CE>DE>EE>#'E># E>$+%E>&$3'E>(>)E>*++E>,7-E>.%/E>0 /1E>2$3E>4,5E>6*7E>8!F9E>: ;E>N X 3((BB F  ^^;;""@@""H .-1"#<@=AMQi Ai  i * i  i :i ;i Ki i i  i X6:!!$2! !H $17 '0'0 '0 / '0  '0 '0`&*3=3F3F3I3I: : : " : # : #: 1:  : D#%)3=3F3F3I3I\ 4\ "\ # \  \ #\ 1\  \ B 3 @33 3  3j : 6;!"!.2! !R'+%)-2!%!%]K]K]K ]K $ ]K#]K'+]K]K]K]K ]KF"&!% PUPU PU  PU  PUl"&!% 1 1  1  1 f)+-(D>"&!% S )S S  S  Sr  U3"U3 U3  U3U3 U3x"&!% )7)7 )7  )7  )7^"&!% &? M&? &?  &?  &?X  j3"-2j3 j3  j3j3'+j3 j3`!%!% @ %@ @  @  @N HH!:H  H\#)-2604#!%"&+/ $$(8<-27;!%!%%MEME ME ' ME 0 ME.MEMEME ME)MEME"ME6ME'+ME 5!ME"#ME$%ME&'ME( )MEf"&!% LULU LU  LUd"&!% '' '  '  'R-2-2^ &(V8=%%04% %TCG<@"&&*26# $&*%)"&+/26 $/3*.#%@@@: @  @ $ @0@@@$@#@ @)@0@@ -!@"(#@$%@&'@( ")@J'+#'(,"&-1/39=$("& $# $&*%)"&*.%V $V! V & V  V+V-V 7V"V VVVV$V #!V" #V$(%V& 'Vx'+(, $  $ $ $ & $ $ LKO::46 D*  MQK.K8IK K4PTB.B9LB !B*)-& '+ & %& $ & ' & T)-& '+ & %& $ & ' & P6     :04# -    & D04"'+#!-    %  2J&*#'(,26-U-U# -U ! -U  -U&-U0-U "-Uf#%)#'(,04/U/U /U # /U ! /U&/U./U "/Ub( 56%)&*# ' "' $ '  ' ( ' 7' TAE0= .)-&* nF&nF$ nF  nFd04"'+2604)-(,)-#-1I,II% I 0 I !. I'I&I'II+I 2IV56/3QU6,6AN6 676@&*)3)<)<)?)?2637LP#'F F # F ' F 0 F 1F JF !F  F V'+(, $&   .56'+(, - $- & -  - 7- b)-C='+ C=%C=$ C= 4 C= 2 C= $C=N)- '+ % $   J04#'&*/3-1#3-3! 3 $ 3 - 3+33 &3j56)-&* 6&6$ 6  676>)-&* :6&:6$ :6 ! :6  :6 :6~)-&* $# !&!$ !  !  ! -!L)-&*-1! /<&/<$ /< + /<  /< #/W $W & W F W '+ W ;W W  <W 7W r 8*X56/3(,&*26/304+/(,+/"':>8<j+j&j$ j 0 j - j.j)j&j)j j%8j6j j7jZLP! !! -H! ! J)-&*379=*.#{##' $$){%{${1 { 7 { ( {{{{!{{"{{ &{z  22 2  2  22>22 2  226!%aa  a  aaa (aaN,0/3#'+/(,MQ8<U#U#) U# - U# ! U#)U#&U#!KU#6U# U#t$(%)GK-2 W#!W## W# E W# '+ W#W# W#r6$FK/6,/6>B/6 /6h)-&*9= #&#$ # 7 #  #P)-&*'6&'6$ '6  '6 # '6'6 '6R F  2"F"" & "L"& % %  % -%X15  %* F*-* . *  ** '*b)-04(,06:0 0 & 0 . 0&030 -0n15  %* 2*-* . *  ** '*^%) y 04(,&*(,$(04  ) ) - -+/)-?C"$(,7;/3 $ $/y"y y - y&y$y&y"y.y y)y '!y"=#y$ %y&&'y($5)y*-+y,-y./y01y2 3y~04:>(,$()6)6- )6 8 )6 & )6")6 )6V6&0404RV# )-). ) P )  )) ')V6(0406 ;?(,$(#'06+06-06 8 06 & 06"06!06 06nHL?C)-?A?A?A E ?A = ?A'?A (?ALHL dd%d E d  d 2dT $   2 :)-(,# &&&& &  &  & .&P=#*= #=6)-(,5=5=0 5=  5=&5=&5= $5=n,=$,= ,= 0 ,=  ,= $,=\ 6$.2"'+# (+( ( % (  ( .(\&*&*-1 % % # % $ % + %  /% NH.&*-1 66 6 # 6 + 6 6@6 66-:6 6(66,36 6&HH+.H .H<04%)26-1 $9>9>'9> - 9> # 9>09>+9> 9> *9>v ?  "% $ IJ>%(>14>BE> >*!$,0 . . ) I J$(&+1;1K1K1V1V17#*A*A *A " *A $ *A/*A/*A*A (*AX2&* $(,3<3<# 3<  3< & 3<3< 3r c SSKJn U"5nU=(d 0nURSS5nUR"U4SU0UD6$![a [S5ef=f![ an[ R"SU35eSnAff=f)aConvert a prompt to OpenAI format. Requires the `langchain_openai` package to be installed. Args: messages (Any): The messages to convert. model_kwargs (Optional[Dict[str, Any]]): Model configuration arguments including `stop` and any other required arguments. Returns: dict: The prompt in OpenAI format. Raises: ImportError: If the `langchain_openai` package is not installed. ls_utils.LangSmithError: If there is an error during the conversion process. r) ChatOpenAIzThe convert_prompt_to_openai_format function requires the langchain_openaipackage to run. Install with `pip install langchain_openai`stopNz#Error converting to OpenAI format: )langchain_openairIrLr__get_request_payloadrr;r)ru model_kwargsrIrrJrs r?convert_prompt_to_openai_formatrN #s( /\F%2L   FD )DQ**8O$O,OO   K    Q%%(KA3&OPPQs"AAA B%A>>BcPSSKJn U=(d 0nURSS5nURSS5nURSS5nU"S X5US .UD6nUR XS 9$![a [S5ef=f![ an[ R"S U35eSnAff=f) aAConvert a prompt to Anthropic format. Requires the `langchain_anthropic` package to be installed. Args: messages (Any): The messages to convert. model_kwargs (Optional[Dict[str, Any]]): Model configuration arguments including `model_name` and `stop`. Returns: dict: The prompt in Anthropic format. r) ChatAnthropiczThe convert_prompt_to_anthropic_format function requires the langchain_anthropic package to run. Install with `pip install langchain_anthropic` model_namezclaude-3-haiku-20240307rJNr)rQrrJ)rJz&Error converting to Anthropic format: r=)langchain_anthropicrPrLr_rLrr;r)rurMrPrQrJrrrs r?"convert_prompt_to_anthropic_formatrS3#s  5 %2L!!,0IJJ   FD )Dy$/GT=IIT--h-BB#   =   $ T%%(Nqc&RSSTs#A$A=$A:= B%B  B%c>^\rSrSrSrSU4SjjrSSSjjrSrU=r$) _FailedAttachmentReaderi[#zHBytesIO that raises an error when read, for failed attachment downloads.c.>[TU]5 Xlgr)rr_error)rrrs r?r _FailedAttachmentReader.__init__^#s  r>c^[R"SUR35URe)NzFailed to download attachment: )r;rrW)rrs r?read_FailedAttachmentReader.readb#s*%%-dkk] ; {{ r>)rW)rr)r@)rrrr ) rSrTrUrVrWrrZrXrrs@r?rUrU[#sRr>rUrc $0nX;aX(aXR5HupEURS5(dMUb[X%S5nOUSn[R"USS9nUR 5 [ R"UR5n[S 0USUUR S5S .D6X4RS5'M U$![a0n [RSUSU 35 [U 5nSn A NmSn A ff=f) zOConvert attachments from the backend database format to the user facing format.z attachment.N presigned_urlT)rzError downloading attachment rr)r]readerrr=)r startswithrr`rr!r2r3contentrrrtrUr5 removeprefix) rr]rattachments_dictrrr8rr^rs r?r^r^h#s4#8/557JC>>-00")'3IJ 1 4#<<>))+H$4$45AOA%*?%;$!&;!7A --m< =8*  4!>se2aSIJ03 4sAC D&D  DcUHnUR5 M g![a$ [RSUR5 MFf=f)z2Close all opened files used in multipart requests.zCould not close file: %sN)rrrrr1)rr>s r?rr#sA  JJL  LL3TYY ?  s*A  A cdURS5RS5(aSUS3$SUS3$)Nrnrz/platform/datasets/rz/v1/platform/datasets/rr)rrs r?rr#s:~~c##E**$ZL ::' |9==r>c`URS5RS5(aSU3$SU3$)z=Construct a platform API path based on the API URL structure.rnrz /platform/z /v1/platform/re)rrxs r?r7r7#s6~~c##E**D6""tf%%r>c@URS5(aU$URS5(aSnU[S5SnO6URS5(aSnU[S5SnO[SU<35eURS5R S5nUR S5R S5nU(d[SU<SU<35eU(dU$USS :XaUS S :XaUSS nOUS SS S /:XaUSS nX4-nUSR S U55-$)NhttprMrLzBapi_url must start with 'http://' or 'https://'. Received api_url=rnzPMust specify non-empty api_url or pathname must be a full url. Received api_url=z , pathname=rapir@rv1c36# UHo(dM Uv M g7frr=)rps r?r!_construct_url..#s1eq11es  )r_rzrurrylstripr)rrrh api_partsrrs r?rr#sB6""*%%#j/+,   I & &#i.*+Q S  s#))#.I%++C0J  !zH; 0   !} R=E !!#2I rs^t} ,!#2I  "E #((1e11 11r>rc[US5(aURUS9$[US5(aURUS9$[S5e)z)Dump model depending on pydantic version. model_dumprrzUnsupported model type)rzrqr TypeError)rrs r?rr#sMul##\::   zz|z44011r>cSSKJn SSKJn SSKJnJn Un[X5(Ga8[URU5(Ga[URSU5(aS[UR5s=::aS::aO U$URnURSnURnURn [Xb5(d3S UR;a#URS n U"S S UR0U D6n[Xb5(aTXh-RSRn U R5V V s0sHupX;dM X_M sn n Ul U"Xg5nU$![a [S5ef=fs sn n f) z/Format the object so its Prompt Hub compatible.r)ChatPromptTemplaterurwrRrrrls_structured_output_formatrur=)rrtrrvlangchain_core.runnablesrxryrLrqrrrzrrrur)objrtrvrxry chain_to_pushr bound_modelrrM output_formatstructured_kwargsrmrns r?rWrW#sk =FNM3)) syy"4 5 5 syy|_ 5 5 SYY $1 $6 3iil !!")) 644-1C1CC'../LMM%PvP-PF f / /!' 6 6q 9 @ @ ".!3!3!5"!59S!5"K  -VAM Q   I   B"sE#; E< E<#E9r)rr,) r~r3rrtrrrrtrztuple[str, str])rrtrr)rr)rzrequests.Sessionrr)rrtrr1rr)rr-r)rrrz float | None)rrrr2)rr ror1rr)rOptional[Union[str, uuid.UUID]]rr)rr|rrrr)rr|rr)rurrMrrr)rrr]rtrr1rzdict[str, AttachmentInfo])rzlist[io.BufferedReader]rr)rrtrr rrt)rrtrxrtrrt)rrtrrtrrt)rrrr)rwrrr)rW __future__rrfrrrAr=rr[rorimportlib.metadatar2rrloggingrXrrhrrrtypingrrrrdcollections.abcrrrrrinspectr pathlibr rr r r rrrrrrurllibrrvr`pydanticrrrrequests_toolbeltrrtyping_extensionsrrurllib3.poolmanagerr urllib3.utilrrrrfrrbrr;langsmith._internalrrr r!rk#langsmith._internal._beta_decoratorr"&langsmith._internal._compressed_tracesr#langsmith._internal._constantsr$r%r&r'langsmith._internal._multipartr(r)r*langsmith._internal._operationsr+r,r-r.r/r0r1r2langsmith._internal._serder3r'langsmith._internal._uuidr4langsmith.schemasr5r6rrr@rNzoneinforOrLopentelemetry.sdk.tracerYr rrvr\rHrCrMrKrGr\r^rUrr_r`rarrbrcrdrerfrgrhri getLoggerrSrrrrkrA parametersrrGrrrsrtr r rprrrrrrr{rr HTTPAdapterrr rNrSr3rUr^rrr7rrrWr=r>r?rs #      PP   )22'#+'':C    A+="(6  $(!.61!5H?      8 $##$<=    , '0B0M0MMS  '     :   3  H  Q3 3>4  .2*. $ JJCG *<@ QV Q  T-99T8eEeETK.2$Q$Q*$Q $QR.2%T%T*%T %TP bjj CG $'2?<>&2D/42.UZ((((...."   s6L L7( M L43L47M  M  MM