WiLcdZddlmZddlZddlZddlZddlmZmZm Z ddl m Z m Z m Z mZmZmZmZmZmZddlmZddlmZmZmZddlmZmZmZe rdd lm Z!ejDd Z#ed d Z$edeZ%edZ&ddZ' d dZ( d!dZ) d"dZ*d#dZ+d$dZ,d%dZ-Gddee$Z.d&dZ/dZ0d'dZ1d(dZ2d)dZ3y)*zwCore utilities for instructor library. This module contains generic utility functions that are not provider-specific. ) annotationsN)AsyncGenerator GeneratorIterable) TYPE_CHECKINGAnyCallableGenericUnionTypeVarcastget_args get_origin)CompletionUsage)ChatCompletionChatCompletionMessageChatCompletionMessageParam) BaseModelValidationError create_modelUsage instructorR_coT) covariantT_Model)boundTcv|jd}|jd}|dk7r|dk7r |||dz}|S|}|S)ai Extract JSON from a string that may contain extra text. The function looks for the first '{' and the last '}' in the string and returns the content between them, inclusive. If no braces are found, the original string is returned. Args: content: The string that may contain JSON Returns: The extracted JSON string {})findrfind)content first_brace last_brace json_contents a/home/sietch6/trending-topics-pipeline/venv/lib/python3.12/site-packages/instructor/utils/core.pyextract_json_from_codeblockr++sT,,s#Ks#JbZ2-{Z!^<   c#nKd}d}d}d}d}g}g}g}|D]} | D]} |s+| dk(r&|j| t|dk(rd}d}g}1t|dkDr| dk7rg}|rG|sE| dk(r|dz }|dk(rd}d}^|dkDrd}| dk(r$d}|jd|j| |r| dk(r|s| }n| d k(r|rd}|j| d}|r,|s*| dk(r|dz }|dk(rd}|D]} | g}d}|dkDrd}|sS| dk(r|jdn<| d k(r7|r5|j|s#|j| |D]} | g}d}B|j| O|rS|rW| dk(s^d}|jd|j| |r|r |D]} | y y y w) aJ Extract JSON from a stream of chunks, handling JSON in code blocks. This optimized version extracts JSON from markdown code blocks or plain JSON by implementing a state machine approach. The state machine tracks several states: - Whether we're inside a code block (```json ... ```) - Whether we've started tracking a JSON object - Whether we're inside a string literal - The stack of open braces to properly identify the JSON structure Args: chunks: An iterable of string chunks Yields: Characters within the JSON object Fr`Tr#r "\r!Nappendlenpop chunks in_codeblockcodeblock_delimiter_count json_started in_string escape_next brace_stackbuffercodeblock_bufferchunkcharcs r*extract_json_from_streamrCDs`,L !LIKK FX$W $DDCK ''-'(A-#'L01-')$%&*ts{#% L3;-2-0A5', 451.2. 3;#'L&&s+MM$'3;{$- IT\i"&KMM$'"'K s{1Q6149+0L%+(&'(%'F+0L! 2Q6451!s{#**3/#)*"MM$/%+(&'(%'F+0L! d# # ""3' d#oW $X$v AG |sE*F5.F52F59L)  ! ! -  N#i.#.$& #&y> <ot4#2#6#6v#>L#v-'6':':62'F (L8(%2*9*=*=i*L(O;( <.C N I$**W%7%7%9/%JKK Jr,ctj|}t|dr2|j}|xstj|}t|dr2|S)zGReturns true if the callable is async, accounting for wrapped callables __wrapped__)inspectiscoroutinefunctionrmr})func is_coroutines r*is_asyncrsP..t4L $ &#Hw'B'B4'H  $ & r,c`|sgSt|}g}d}t|dtd|D])\}}t|j ddt r'd}n|r|dkDrt d|ddD}|D]}|j dd }|j dd}|st|t rd |d g}|ra||d dk(rV|r|d dxxd |z cc<bt|tr|d dj||d dj||j||d|S)a Merge consecutive messages from the same role into a single message. This optimized version pre-allocates the result list and minimizes operations. Args: messages: List of message dictionaries to merge Returns: List of merged message dictionaries TN r&reFc3\K|]$}t|jddt&yw)r&reN)rIrqro.0ms r* z-merge_consecutive_messages..s"WA*QUU9b%93?Ws*,rguserrk)rjrkr"z rf) r4 enumerateminrIrqroalllistextendr3) messages message_count new_messages flat_string_irrtrg new_contents r*merge_consecutive_messagesrsi  MML K8$A!%% 2.4K  }r)WRTRUWW H{{66*kk)R0 z+s;$*K@AK DL$4V$<<R +k]/CC+k40 $Y/66{C!$Y/66{C   + F G-H0 r,c eZdZdZddZddZy) classpropertyaDescriptor for class-level properties. Examples: >>> from instructor.utils import classproperty >>> class MyClass: ... @classproperty ... def my_property(cls): ... return cls >>> assert MyClass.my_property c||_yN cproperty)selfmethods r*__init__zclassproperty.__init__s r,c$|j|Srr)rinstanceclss r*__get__zclassproperty.__get__s~~c""r,N)rzCallable[[Any], R_co]returnNone)robjectrz type[Any]rr)__name__ __module__ __qualname____doc__rrr,r*rrs  #r,rcn|sdgS|jdd}t|tr|r|SdgS||gSdgS)a Extract content from a message and ensure it's returned as a list. This optimized version handles different message formats more efficiently. Args: message: A message in ChatCompletionMessageParam format Returns: The message content as a list rer&)rqrIr)rtr&s r*get_message_contentrsS t kk)R(G'4 !w+t+*G 33 33r,crttdstjt_d}|t_y)aDisable URLs in Pydantic ValidationError messages. This function monkey-patches Pydantic's ValidationError.__str__ method to prevent URLs from being included in error messages. This is necessary because Pydantic reads the PYDANTIC_ERRORS_INCLUDE_URL environment variable at import time, not at validation time, so setting it later has no effect. The function works by storing the original __str__ method and replacing it with a version that filters out URLs from the error message. _original_strctj|}g}|jdD]}d|vs|j|dj |S)N zhttps://errors.pydantic.dev)rrsplitr3join)routputlineslines r*__str__z+disable_pydantic_error_url..__str__%sV ..t4LL& #D,D8 T" #yyr,N)rmrrr)rs r*disable_pydantic_error_urlrs+ ?O 4(7(?(?%  &Or,cbt|txrt|txr t |dS)N__annotations__)rIrj issubclassrprm)rs r* is_typed_dictr4s03 , sD ! , C* +r,cddlm}||S)z5Check if a type is a simple type that can be adapted.r)is_simple_type)instructor.dsl.simple_typer)typehint_is_simple_types r*rr<sL 8 $$r,c8|yt|}|turVt|rKt|}|r|dnd}d d}|||rt}n"ddlm}|j |}t|}t|r`t|dd}t|di}tttt|fi|jD cic] \}} || df c} }}t|}|tthvrdd lm} t|}|r|d t#d |d} t| r[ttttt| ddfit| dijD cic] \}} || df c} }} | ttt| }t|rddlm}|j |}d d lm} m} t+j,|rt/|| s | |}|St+j,|s| |}|Scc} }wcc} }w)a Prepares the response model for use in the API call. This function performs several transformations on the input response_model: 1. If the response_model is None, it returns None. 2. If it's a simple type, it wraps it in a ModelAdapter. 3. If it's a TypedDict, it converts it to a Pydantic BaseModel. 4. If it's an Iterable, it wraps the element type in an IterableModel. 5. If it's not already a subclass of OpenAISchema, it applies the openai_schema decorator. Args: response_model (type[T] | None): The input response model to be prepared. Returns: type[T] | None: The prepared response model, or None if the input was None. Nrctj|rt|tryt |t uxrt dt|DS)NTc3jK|]+}tj|xrt|t-ywr)r~isclassrrrs r*rzAprepare_response_model.._is_model_type..bs-2DE"?z!Y'??2s13)r~rrrrr rr)ts r*_is_model_typez.prepare_response_model.._is_model_type_sIq!jI&>a=E)c2IQRS2/ r,) ModelAdapterrTypedDictModelr.) IterableModelzGresponse_model must be parameterized, e.g. list[User] or Iterable[User]) OpenAISchema openai_schema)rrrbool)rrrrrr__class_getitem__rrHr rjrritemsrinstructor.dsl.iterabler ValueErrorprocessing.function_callsrrr~rr)response_modeloriginargsinnerrr model_namerkvriterable_element_classrrs r*prepare_response_modelrCs-"  'F~.8'QT   !6F ?*;;NKN/F^$^Z9IJ n.?D  O  +<;+<+<+>?41a1q#h;?   'F (D!!9'tAwY "&a / 0%)Y2J@PQ%,24Er%%'  AqAs8  & "'tDO=S'TUn%;&77GH~&z.,/W&~6 __^ ,&~6 ]@(s ?H&H)r&rorro)r7z Iterable[str]rzGenerator[str, None, None])r7AsyncGenerator[str, None]rr)r[zT_Model | Noner\zOpenAIUsage | AnthropicUsagerzT_Model | ChatCompletion | None)rtrrr)rzCallable[..., Any]rr)rlist[dict[str, Any]]rr)rtrrz list[Any])rr)rztype[T]rr)rtype[T] | Nonerr)4r __future__rr~rrloggingcollections.abcrrrtypingrrr r r r r rr openai.typesrrJopenai.types.chatrrrpydanticrrrrSrrb getLoggerrYrrrr+rCrErcr{rrrrrrrrrr,r*rs # ??   8 >=7   < (v& )9 - CL2~ ~~B~ %~~B66-6%6rB9x#GDM#*46&>%dr,