WiddlZddlZddlZddlmZmZmZddl m Z ddl m Z ddl mZGddeZ ddej d ej d eed ed ed eef dZy)N)ListMappingTuple) csr_matrix)cosine_similarity)BaseRepresentationc eZdZdZd dedefdZdejde de e e e e effde e e e e efffd Zy ) MaximalMarginalRelevanceaCalculate Maximal Marginal Relevance (MMR) between candidate keywords and the document. MMR considers the similarity of keywords/keyphrases with the document, along with the similarity of already selected keywords and keyphrases. This results in a selection of keywords that maximize their within diversity with respect to the document. Arguments: diversity: How diverse the select keywords/keyphrases are. Values range between 0 and 1 with 0 being not diverse at all and 1 being most diverse. top_n_words: The number of keywords/keyhprases to return Usage: ```python from bertopic.representation import MaximalMarginalRelevance from bertopic import BERTopic # Create your representation model representation_model = MaximalMarginalRelevance(diversity=0.3) # Use the representation model in BERTopic on top of the default pipeline topic_model = BERTopic(representation_model=representation_model) ``` diversity top_n_wordsc ||_||_y)N)r r )selfr r s h/home/sietch6/trending-topics-pipeline/venv/lib/python3.12/site-packages/bertopic/representation/_mmr.py__init__z!MaximalMarginalRelevance.__init__'s"& documentsc_tf_idftopicsreturnc|jtjd|Si}|jD]\}}|Dcgc]}|d } }|j | dd} |j dj | ddj dd} t| | | |j|j}||D cgc]\}} ||vs || fc} }||<|Scc}wcc} }w) a7Extract topic representations. Arguments: topic_model: The BERTopic model documents: Not used c_tf_idf: Not used topics: The candidate topics as calculated with c-TF-IDF Returns: updated_topics: Updated topic representations zhMaximalMarginalRelevance can only be used BERTopic was instantiatedwith the `embedding_model` parameter.rwordF)methodverbose ) embedding_modelwarningswarnitems_extract_embeddingsjoinreshapemmrr r ) r topic_modelrrrupdated_topicstopic topic_wordsrwordsword_embeddingstopic_embeddingvalues rextract_topicsz'MaximalMarginalRelevance.extract_topics+s$  & & . MM8 M"(,,. l E;)45T!W5E5)==eF\a=bO)==chhuoV\fk=ltt2O  KGMUm$k{tUW[_jWjdE]$kN5 ! l6%ls C C# C#Ng? )__name__ __module__ __qualname____doc__floatintrpd DataFramerrstrrrr-rrr r s{8'%'C'(<<( ( T%U "3445 ( d5e,-- . (rr doc_embeddingr*r)r top_nrct||}t|}tj|g}tt |Dcgc] }||dk7s |} }t|dz D]} || ddf} tj || dd|fd} d|z | z|| j ddzz } | tj| }|j|| j||Dcgc]}|| c}Scc}wcc}w)aMaximal Marginal Relevance. Arguments: doc_embedding: The document embeddings word_embeddings: The embeddings of the selected candidate keywords/phrases words: The selected candidate keywords/keyphrases diversity: The diversity of the selected embeddings. Values between 0 and 1. top_n: The top n items to return Returns: List[str]: The selected keywords/keyphrases rrN)axisr) rnpargmaxrangelenmaxr#appendremove)r:r*r)r r;word_doc_similarityword_similarity keywords_idxicandidates_idx_candidate_similaritiestarget_similaritiesr$mmr_idxidxs rr$r$Vs*,O]K'8OII123L!&s5z!2KAa<?6JaKNK 519  '"5^Q5F!G ff_^%DQ _%U\]^9} 66EXE`E`acefEg9gg 30 G$g& '#/ /3E#J //!L 0s DD1 Dr.)rnumpyr>pandasr6typingrrr scipy.sparsersklearn.metrics.pairwiserbertopic.representation._baserr ndarrayr8r4r5r$r9rrrVs''#6<I1I` *0::*0ZZ*0 9*0 *0  *0  #Y *0r