Wi#6NUddlmZddlmZmZmZmZddlmZddl m Z ddl m Z m Z ddlmZddlmZmZmZej(eZidid id id id id idididididididididididiZeeeee ffed<edeeefdeeeffdZedeedeefdZdeeeefeefdeeeefeeffdZGdd ZGd!d"eZGd#d$eZGd%d&eZ ed'(d)eded*fd+Z!d,ed-edefd.Z"y')/) lru_cache)AnyOptionalUnionoverload) constants)InferenceProviderMapping) MimeBytesRequestParameters)ChatCompletionInputMessage)build_hf_headers get_tokenloggingcerebrascohereclarifaizfal-aiz fireworks-aigroqz hf-inference hyperbolicnebiusnscaleovhcloud replicate sambanovascalewaytogether wavespeedzzai-org!HARDCODED_MODEL_INFERENCE_MAPPINGobjreturncyNrs x/home/sietch6/trending-topics-pipeline/venv/lib/python3.12/site-packages/huggingface_hub/inference/_providers/_common.py filter_noner%-s8;cyr!r"r#s r$r%r%/s.1r&cft|trEi}|jD].\}}| t|ttfr t |}|||<0|St|tr1|Dcgc]%}t|ttfr t |n|'c}St dt |cc}w)NzExpected dict or list, got ) isinstancedictitemslistr% ValueErrortype)rcleanedkvs r$r%r%3s#t"$IIK DAqy!dD\*NGAJ   #tNQR*Qt "= A1DRR 249+> ??Ss**B.ceZdZdZdedededdfdZddd ed eeefd ed eed eedeeeefde fdZ d de e efdee defdZ d eedefdZd eedefdZd eeefdeeeefdeedeeeffdZd ed edeeeffdZd ededefdZd edefdZded edefdZd ed ededeefdZd ed ededeedeef dZy)!TaskProviderHelperz.Base class for task-specific provider helpers.providerbase_urltaskrNc.||_||_||_yr!)r4r6r5)selfr4r5r6s r$__init__zTaskProviderHelper.__init__Gs     r&) extra_payloadinputs parametersheadersmodelapi_keyr:c|j|}|j|}|j||}|j||j}|j |||} | t | t|xsi} |j||||} | | td| | td|j|| | } t||j|j| | | S)z Prepare the request to be sent to the provider. Each step (api_key, model, headers, url, payload) can be customized in subclasses. )provider_mapping_infoz8Both payload and data cannot be set in the same request.z2Either payload or data must be set in the request.)urlr6r>jsondatar=) _prepare_api_key_prepare_mapping_info_prepare_headers _prepare_url provider_id_prepare_payload_as_dictrecursive_merger%_prepare_payload_as_bytesr-_normalize_headersr r6) r8r;r<r=r>r?r:rArBpayloadrDnormalized_headerss r$prepare_requestz"TaskProviderHelper.prepare_requestLs ''0!% : :5 A''9)>)J)JK// Zo/p  %g{=;NB/OPG--fjBWYfg  4#3WX X ?t|QR R"44WgtL '33&   r&responserequest_paramsc|S)z Return the response in the expected format. Override this method in subclasses for customized response handling.r")r8rQrRs r$ get_responsezTaskProviderHelper.get_responses r&cT| t}|td|jd|S)zZReturn the API key to use for the request. Usually not overwritten in subclasses.z)You must provide an api_key to work with z$ API or log in with `hf auth login`.)rr-r4r8r?s r$rEz#TaskProviderHelper._prepare_api_keys: ?kG ?;DMM?Jno r&c |td|jdtj|jij|rt|j|Sd}t |D]}|j|jk(s|}n|td|d|jd|j |j k7r6td|d|j d|jd|j d |j d k(r&tjd|d |jd |j d k(r&tjd |d|jd|S)zbReturn the mapped model ID to use for the request. Usually not overwritten in subclasses.Nz+Please provide an HF model ID supported by .zModel z is not supported by provider z is not supported for task z and provider z. Supported task: stagingz! is in staging mode for provider z. Meant for test purposes only.errorz,Our latest automated health check on model 'z' for provider 'z<' did not complete successfully. Inference call might fail.) r-r4rget!_fetch_inference_provider_mappingr6statusloggerwarning)r8r>provider_mappingmappings r$rFz(TaskProviderHelper._prepare_mapping_infos =J4==/YZ[\ \ - 0 0 C G G N4T]]CEJ J8? G4==0#*    #veW,J4==/YZ[\ \  DII -:499+^TXTaTaSbc##3#8#8"9<   " "i / NN@Ono   " "g - NN>ugEUVZVcVcUde--  r&rNrDc|jDcic]\}}| |j|}}}|jd&||j|j|d<|S|d|d<|Scc}}w)zzNormalize the headers to use for the request. Override this method in subclasses for customized headers. z content-typezapplication/json)r+lowerr[ mime_type)r8r=rNrDkeyvaluerOs r$rMz%TaskProviderHelper._normalize_headerssDK==?hZS%V[Vgciik50hh  ! !. 1 9DNN$>59^^">2"!$5G">2!! is A.A.c"it||S)zwReturn the headers to use for the request. Override this method in subclasses for customized headers. )token)r )r8r=r?s r$rGz#TaskProviderHelper._prepare_headerss >"1=W==r& mapped_modelc|j|}|j||}|jdd|jdS)zVReturn the URL to use for the request. Usually not overwritten in subclasses./)_prepare_base_url_prepare_routerstriplstrip)r8r?rir5routes r$rHzTaskProviderHelper._prepare_urlsK))'2##L'://#&'qc):(;<>s>tCH~>=C=s=s= ! ! !3'+D\ $    8   ~  )  r&r3c neZdZdZdedeffd ZdededefdZd eee e fd e d e de e fd Z xZS) BaseConversationalTaskz Base class for conversational (chat completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/chat r4r5c*t|||dy)Nconversationalr4r5r6superr9r8r4r5 __class__s r$r9zBaseConversationalTask.__init__ s (XDTUr&rir?rcy)Nz/v1/chat/completionsr"rzs r$rmz%BaseConversationalTask._prepare_routes%r&r;r<rAc<td|i|d|jiS)Nmessagesr>r%rIr|s r$rJz/BaseConversationalTask._prepare_payload_as_dicts' Ji*igG\GhGhijjr&)r~rrrrr9rmr,rr*r r rrJ __classcell__rs@r$rrs| VVV&3&&&kU4!;;<=kk 8 k $ kr&rcHeZdZdZfdZdedefdZdeedefdZ xZ S)AutoRouterConversationalTaska Auto-router for conversational tasks. We let the Hugging Face router select the best provider for the model, based on availability and user preferences. This is a special case since the selection is done server-side (avoid 1 API call to fetch provider mapping). c(t|ddy)Nautozhttps://router.huggingface.co)r4r5r)r8rs r$r9z%AutoRouterConversationalTask.__init__$s &3RSr&r?rcR|jds td|jS)rrrsz>Cannot select auto-router when using non-Hugging Face API key.)rtr-r5rVs r$rlz.AutoRouterConversationalTask._prepare_base_url's( !!%(]^ ^== r&r>c<| tdtd||ddS)z In auto-router, we don't need to fetch provider mapping info. We just return a dummy mapping info with provider_id set to the HF model ID. zPlease provide an HF model ID.rliver)r4 hf_model_id providerIdr]r6)r-r )r8r>s r$rFz2AutoRouterConversationalTask._prepare_mapping_info1s1 ==> >'!   r&) r~rrrr9rrlrr rFrrs@r$rrs9T!!! 8C= =U r&rc ^eZdZdZdedeffd ZdededefdZd ed ed e de efd Z xZ S) BaseTextGenerationTaskz Base class for text-generation (completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/completions r4r5c*t|||dy)Nztext-generationrrrs r$r9zBaseTextGenerationTask.__init__Hs (XDUVr&rir?rcy)Nz/v1/completionsr"rzs r$rmz%BaseTextGenerationTask._prepare_routeKs r&r;r<rAc<td|i|d|jiS)Npromptr>rr|s r$rJz/BaseTextGenerationTask._prepare_payload_as_dictNs'Hfg gGEZEfEfghhr&) r~rrrrr9rmrr*r rrJrrs@r$rrBsf WWW!3!!!ii'+iD\i $ir&rN)maxsizer>r c|ddlm}|j|dg}|j}|t d||S)z; Fetch provider mappings for a model from the Hub. r)HfApiinferenceProviderMapping)expandz$No provider mapping found for model )huggingface_hub.hf_apir model_infoinference_provider_mappingr-)r>rrur`s r$r\r\TsM - 7  e-G,H  ID66?wGHH r&dict1dict2c i||jDcic]>\}}|||vr2t||trt|trt|||n|@c}}Scc}}wr!)r+r)r*rK)rrrerfs r$rKrKbsx    $kkm  U u E#J!=*UTXBY!sU3    sAA)# functoolsrtypingrrrrhuggingface_hubrrr !huggingface_hub.inference._commonr r :huggingface_hub.inference._generated.types.chat_completionr huggingface_hub.utilsr rr get_loggerr~r^rr*r__annotations__r%r,r3rrrr\rKr"r&r$rsE11%;JaFF   H % UU bUU bUBU BUBU"U  b!U" b#U$%U&'U()U*+U,-U./U0r1U!4T#7O2O-P(P#Q8 ;T#s(^;S#X; ; 1T#Y1491 1@U4S>4945@%S#XPTUXPY@Y:Z@"@@Fk/k*# #9# Li/i$ 4 S T:T5U   4   r&