/~hqdZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddlmZmZe e j"Zd dZd dZdd Zdd Zdd Zy)z"Better tokenizing for coverage.py.) annotationsN)Iterable)env)TLineNoTSourceTokenLinesc #Kd}d}d}|D]:\}}\}}\}} } ||k7r|r|jdrd} |jdrd} nr|tjk(r7|jdr#|jdj|rd} n+d} n(tj d k\r|tj k(rd} | rBt|jd d d z } tjd d|| f|| dzf|| }|tjtjfvr|}tj||||f|| f| |}=yw)aBReturn all physical tokens, even line continuations. tokenize.generate_tokens() doesn't return a token for the backslash that continues lines. This wrapper provides those tokens so that we can re-create a faithful representation of the original source. Returns the same values as generate_tokens() Nz\ T\Fz \   i) endswithtokenSTRINGrstripsys version_infoFSTRING_MIDDLElensplittokenize TokenInfoNEWLINENL) toks last_line last_lineno last_ttextttypettextslinenoscolelinenoecolltextinject_backslashccols \/var/www/peopleoo.sandbox-dev.co.uk/venv/lib/python3.12/site-packages/coverage/phystokens.py _phys_tokensr.si!IKJAE2=uow ' !Y//7 $( &&t,',$ell*!**62!((1:::F,0(,1(%%0Ue>R>R5R',$#yt4R89A=D",,v $'46):! I ))8;;7 7J  $QVWW e2sEE ct}tjtj|D]}tj dk\rlt |tjrR|j|j|jD]'}|j|jj)tj dk\st |tjs|j|j|S)zCHelper for finding lines with soft keywords, like match/case lines.r r ) setastwalkparserr isinstanceMatchaddlinenocasespattern TypeAlias)sourcesoft_key_linesnodecases r-find_soft_key_linesrA[s#&5N6*+,   w &:dCII+F   t{{ +  8""4<<#6#67 8    (Zcmm-L   t{{ + , c#<Ktjtjtjtj h}g}d}|j djdd}t|}tjjr t|}n t}t|D]d\}}\}} \} } } d} tj d|D]3} | dk(r |g}d}d}n| dk(rd}n||vrd}n | r | |kDr|j#d d | |z zfd} tj$j'|d j)d d }|tj*k(rt-j.|rd}nyt0j2dk\rftjjrLt-j4|r7t7|dk(rd}nt7|dk(r|ddd k(rd}nd}|r||vrd}|j#|| fd}d} 6sc| }g|r|y y w)aGenerate a series of lines, one for each line in `source`. Each line is a list of pairs, each pair is a token:: [('key', 'def'), ('ws', ' '), ('nam', 'hello'), ('op', '('), ... ] Each pair has a token class, and the token text. If you concatenate all the token texts, and then join them with newlines, you should have your original `source` back, with two differences: trailing white space is not preserved, and a final line with no newline is indistinguishable from a final line with a newline. rz rTz( )Fr ws xxNr keyr0r)rINDENTDEDENTrrr expandtabsreplacegenerate_tokensr PYBEHAVIOR soft_keywordsrAr2r.rerappendtok_namegetlowerNAMEkeyword iskeywordrr issoftkeywordr)r= ws_tokenslinecoltokgenr>r$r%sliner'_r) mark_startpartmark_end tok_classis_start_of_lines r-source_token_linesrdjs u||U]]HKKHI"$D C   q ! ) )&$ 7F V $F ~~##,V45A&5I&1umudYa HHVU+" Dt|   )# $*KKsdSj'9 :;!&J$--11%>DDFrJ EJJ&((/$) ))W4>>77GrzsY)"  $5h(( ) ?D EP. 1rB