Wi8 d dl Zd dlZd dlmZmZmZ d dlm Z d dl
m Z d dl
mZ d dlmZ d dlmZ d dlmZ d dlmZ dded ee dz d
edz d ed eeef d
edededej6 dee gej8 f dz dee ge f dz dedej: fdZ ddej6 de dee gej8 f dee ge f ded edeee fdZ y) N)CallableListUnion)
csr_matrix) hierarchycosine_similarity)select_topic_representation)validate_distance_matrix orientationtopics top_n_topics
use_ctfidf
custom_labelstitlewidthheighthierarchical_topicslinkage_functiondistance_functioncolor_thresholdreturnc
! d |
d }
| j }
|
j |
j dk7 ddf }
| t | }nL|'t |
j j d| }n#t |
j j }t t | j
j }t j |D cg c] }|j | c} }t | j | j | d | !| Bt | t |
j j k( rt | | ! |
|| }nd} !fd}t! j" !|||
|| }|d k( rd
nd }t% |t& r|j( | d D cg c]# }t' | dgg| j* | | z % }}|D cg c]' }d
j- |dd D cg c] }|d c} ) }}}|D cg c] }t | dk r|n|dd dz }}n| j. H|rF|j( | d D cg c]* }| j. |t1 | | j2 z , }}n|j( | d D cg c]9 }t' |t1 | dgg| j5 |t1 | ; }}|D cg c]' }d
j- |dd D cg c] }|d c} ) }}}|D cg c] }t | dk r|n|dd dz }}|j7 dd| dddt9 dd dt9 dd d |d k( r|j7 d d!t | z z |t9 d"|# $ t; |d% D cg c] }|d& j; d'z c} }t= |d% D cg c] }|d& j= d'z
c} }|j7 t9 ||g( ) n.|j7 d d!t | z z |t9 d"|# * | d+D ] } |d k( rd,nd&}|j> D cg c] } | d- s | | | dkD s| d, | } } |j> D cg c] } | d- s | | | dkD s| d& | } } |j> D cg c] } | d- s | | | dkD s| d- | } } |jA tC jD | | d.| d-d/d01 |S c c}w c c}w c c}w c c}}w c c}w c c}w c c}w c c}w c c}}w c c}w c c}w c c}w c c} w c c} w c c} w )2ap Visualize a hierarchical structure of the topics.
A ward linkage function is used to perform the
hierarchical clustering based on the cosine distance
matrix between topic embeddings (either c-TF-IDF or the embeddings from the embedding model).
Arguments:
topic_model: A fitted BERTopic instance.
orientation: The orientation of the figure.
Either 'left' or 'bottom'
topics: A selection of topics to visualize
top_n_topics: Only select the top n most frequent topics
use_ctfidf: Whether to calculate distances between topics based on c-TF-IDF embeddings. If False, the embeddings
from the embedding model are used.
custom_labels: If bool, whether to use custom topic labels that were defined using
`topic_model.set_topic_labels`.
If `str`, it uses labels from other aspects, e.g., "Aspect1".
NOTE: Custom labels are only generated for the original
un-merged topics.
title: Title of the plot.
width: The width of the figure. Only works if orientation is set to 'left'
height: The height of the figure. Only works if orientation is set to 'bottom'
hierarchical_topics: A dataframe that contains a hierarchy of topics
represented by their parents and their children.
NOTE: The hierarchical topic names are only visualized
if both `topics` and `top_n_topics` are not set.
linkage_function: The linkage function to use. Default is:
`lambda x: sch.linkage(x, 'ward', optimal_ordering=True)`
NOTE: Make sure to use the same `linkage_function` as used
in `topic_model.hierarchical_topics`.
distance_function: The distance function to use on the c-TF-IDF matrix. Default is:
`lambda x: 1 - cosine_similarity(x)`.
You can pass any function that returns either a square matrix of
shape (n_samples, n_samples) with zeros on the diagonal and
non-negative values or condensed distance matrix of shape
(n_samples * (n_samples - 1) / 2,) containing the upper
triangular of the distance matrix.
NOTE: Make sure to use the same `distance_function` as used
in `topic_model.hierarchical_topics`.
color_threshold: Value at which the separation of clusters will be made which
will result in different colors for different clusters.
A higher value will typically lead in less colored clusters.
Returns:
fig: A plotly figure
Examples:
To visualize the hierarchical structure of
topics simply run:
```python
topic_model.visualize_hierarchy()
```
If you also want the labels visualized of hierarchical topics,
run the following:
```python
# Extract hierarchical topics and their representations
hierarchical_topics = topic_model.hierarchical_topics(docs)
# Visualize these representations
topic_model.visualize_hierarchy(hierarchical_topics=hierarchical_topics)
```
If you want to save the resulting figure:
```python
fig = topic_model.visualize_hierarchy()
fig.write_html("path/to/file.html")
```
Nc dt | z
S )N r xs h/home/sietch6/trending-topics-pipeline/venv/lib/python3.12/site-packages/bertopic/plotting/_hierarchy.pyz%visualize_hierarchy..j s a*;A*>&> c 2 t j | dd S )NwardT)optimal_ordering)schlinkager s r r z%visualize_hierarchy..m s S[[FT%R r r ) topic_modelr
embeddingsr r r r c B t | j d S )Nr )r shape)r r r( s r r z%visualize_hierarchy.. s# &>?PQR?SU_UeUefgUh&i r )r distfun
linkagefun hovertextr leftyaxisxaxisticktext_ z...z#ECEFF1 plotly_whiteg ?centertop Black)sizecolor)textr xanchoryanchorfontwhite Rockwell)bgcolor font_size font_family) plot_bgcolortemplater
hoverlabel array)tickmoder1 )r r r/ datay )range)r/ )r r r0 )r r r= blackmarkersF)r rO marker_colorr- hoverinfomode
showlegend)#get_topic_freqlocTopiclistsortedto_list
get_topicskeysnprL indexr
c_tf_idf_topic_embeddings_len_get_annotationsffcreate_dendrogram
isinstancestrlayouttopic_aspects_joincustom_labels_int _outliers get_topic
update_layoutdictmaxminrN add_tracegoScatter)"r' r r
r r r r r r r r r r freq_df
all_topicstopicindices annotationsdistance_function_vizfigaxisr
new_labelslabelslabeltracey_maxy_minrb rN xsysr- r( s" ` @r visualize_hierarchyr s r > R ((*Gkk'--2-q01G
f !
--/
>?
--/0 [ 3 3 5 : : <=>JhhVDE
((/DEG -[-B-BKDaDacmnopqJ
&3v;#gmm>S>S>U:V+V&# 3! / -#'
j
%# '
C "V+7D-%UXU_U_`dUefpUq
PQc!fd^ { 9 9- H K K
R\\vchhfRaj AUa AB\
\T^_5s5zBeE#2J4FF_
_ # # /MX[XbXbcgXhisXt
STK & &vc!f~ 8M8M'M N
\_[e[efj[klv[w
VWc&Q.!4
( Q;+@+@A +P Q
R\\vchhfRaj AUa AB\
\T^_5s5zBeE#2J4FF_
_ g b0
2:N f "s6{*+*=
s6{CeU3Z^^%)CDs6{CeU3Z^^%)CD E5> :; c&k)**=
& E%/3SD/2xxdtDLTRVZX]M^abMb$s)E"dBd/2xxdtDLTRVZX]M^abMb$s)E"dBd9<nd6lW[\`WabgWhklWlf e,nIn MM
!( ' $" $
" JQ EH
B\_
B\_2 DC edns T 4(T$#T.: T) T. T4/T9!>T>&U= U U U, U U+
U 6 U
U
U"' U"3
U"
U' U'$
U')T.Ur( c 2 |j |j dk7 ddf } || }t ||j d } || } t j
| |d }
t
j dt |
d dz dz d } t t |
d | } t }
| j D ] \ }}|g|
|<
t t |j |j }g }t |
d D ] \ }}|
|d }|
|d
}t | d k( rt |t r@|d d d j# t% t | j& | |d dd
z }n| j( "|r | j( |d | j* z }nd j# | j- |d D cg c] \ }}| c}}dd }n[|j D ]H \ }}t/ | t/ | k( s |j |j |k( df j0 d }J t | d k( rt |t r@|d d d j# t% t | j& | |d dd
z }n| j( "|r | j( |d | j* z }nd j# | j- |d D cg c] \ }}| c}}dd }n[|j D ]H \ }}t/ | t/ | k( s |j |j |k( df j0 d }J |j3 ddg |d |d
z d
z }||z |
|< |S c c}}w c c}}w )a Get annotations by replicating linkage function calculation in scipy.
Arguments:
topic_model: A fitted BERTopic instance.
hierarchical_topics: A dataframe that contains a hierarchy of topics
represented by their parents and their children.
NOTE: The hierarchical topic names are only visualized
if both `topics` and `top_n_topics` are not set.
embeddings: The c-TF-IDF matrix on which to model the hierarchy
linkage_function: The linkage function to use. Default is:
`lambda x: sch.linkage(x, 'ward', optimal_ordering=True)`
NOTE: Make sure to use the same `linkage_function` as used
in `topic_model.hierarchical_topics`.
distance_function: The distance function to use on the c-TF-IDF matrix. Default is:
`lambda x: 1 - cosine_similarity(x)`.
You can pass any function that returns either a square matrix of
shape (n_samples, n_samples) with zeros on the diagonal and
non-negative values or condensed distance matrix of shape
(n_samples * (n_samples - 1) / 2,) containing the upper
triangular of the distance matrix.
NOTE: Make sure to use the same `distance_function` as used
in `topic_model.hierarchical_topics`.
orientation: The orientation of the figure.
Either 'left' or 'bottom'
custom_labels: Whether to use custom topic labels that were defined using
`topic_model.set_topic_labels`.
NOTE: Custom labels are only generated for the original
un-merged topics.
Returns:
text_annotations: Annotations to be used within Plotly's `ff.create_dendogram`
TopNr T)r no_plotrP leaves
icoord r r2 rR Parent_Name )rZ r r r* r$
dendrogramra arangere rs zipitems Parent_IDTopics enumerateri rj rm nextrl rn rp rq setvaluesappend) r' r r( r r r r dfXZPx_ticksx_topic
topic_valskeyval parent_topictext_annotationsrb r fst_topic
scnd_topicfst_namewordr2 value scnd_namer7 s r rf rf s R
!4!@!@E!I1!L MB *%A J$4$4Q$78A A qk4@A ii3q{+b014b9G3q{G,-GJMMO S %
3 BLL"))45L !!H+. #4 u uQx( a)
y>Q -- 'l^1-k 8 8 G RS UVWXZYZ[1 ++7M&55il[EZEZ6Z[ 889N9NyYZ|9\$]gdAT$]^`_`$ab * 0 0 2
T
Uu:Y / !vvbllc&9=&H I P PQR SH
T z?a -- )!}oQ /#((k 8 8 G
ST
VWXY[Z[\3 ++7M ' 6 6z!}{G\G\7\ ] HH+:O:OPZ[\P]:^%_wtQd%_`bab%c d * 0 0 2
U
Uu:Z 0 "r||s':M'I J Q QRS TI
U 2r9 =>(U1X%* & 3
6G#4J 3 %^ &`s N
* N ) r. NNTFz Hierarchical Clusteringi iX NNNr )F) numpyra pandaspdtypingr r r scipy.sparser
scipy.clusterr r$ sklearn.metrics.pairwiser bertopic._utilsr
plotly.graph_objects
graph_objectsrw plotly.figure_factoryfigure_factoryrg r rj ro bool DataFramendarrayFigurer rf r r r s ( ( # * 6 7 ! " 4
# #&+1(,BFCG SS
I S * S
S s#
S S S
S S
|RZZ 784?S j 89D@S S YY Sz d d d
|RZZ 78 d
j 89 d
d d
$s)_d r