B b@sddlmZddlZddlmmZeedddZej dddZ d d Z ej d d d Z eeej edddZ ej dddZdS))ListN)phraselgcCsV|dkrtd}n|dkr(td}ntd|rRdddd||DDSgS) zTokenize an utterance based on specified word-forming characters. Parameters: phrase: the string to be tokenized lg: the language whose word-forming characters are to be used in tokenization Return list of tokens tcaz([^0-9A-Za-z\+#_])mtou"([^A-Za-zäëöü'`ꞌꞋ'‘’])z/Language either not given or an invalid string.cSsg|] }|r|qSr).0jrr//home/sunny/Documents/lx/flexible/flexible02.py sztokenize..cSsg|] }|qSr)strip)rirrr r s)recompile ValueErrorsplit)rrZnon_word_formingrrr tokenizes   r)elc Cs(td|jdt|jd|jdt|dS)zPrint the tag, attributes, text, and number of children of an ET.Element Parameters: el: Element to be printed zTag:z Attrs:z Text:z No. of Children:N)printtagstrattribtextlen)rrrr print_el_infosrcCsttdddd}t|dd}ddt||}|dd d |d d d |d d d |d d d |d d}tdddt||S)zGenerate FLEx guid based on offset defined in offset.txt Increments offset upon use. Return guid in format [0-f]{8}-([0-f]{4}-){3}[0-f]{12} z offset.txtr)modeN0 - w)intopenreadhexrwriter)Z global_offsetZ new_guid_numZ new_guid_strZnew_guidrrr generate_guid#s Lr,)eaf_rootcCsdd|dDS)zGet time IDs and values from an EAF file Parameters: eaf_root: is the root element of an EAF object parsed through ElementTree Return a dictionary of time ID and value pairs cSsi|]}|jd|jdqS)Z TIME_VALUEZ TIME_SLOT_ID)r)rr rrr >sztime_values..z .//TIME_SLOT)findall)r-rrr time_values2s r0) tokenized_utt phrase_elrc Cstd}x|D]}tjddtid}|dkrjtd}||rJd}nd}tjd |d |d d}nJ|d krtd }||rd}nd}tjd |d |d d}ntd||_| || |qW| |dS)aPopulate a phrase element with a tokenized utterance *tokenized_utt* is a list with tokens from an utterance *phrase_el* is the element that will be the parent to the words added (below the words el in the phrase_el will be the items w/ translations and notes) *lg* is the language whose word-forming characters are to be used in tokenization Makes changes in place (returns nothing) wordswordguid)rrz([^0-9A-Za-z\+#_])punctZtxtitemz{}-fonipa-x-etic)typelangru"([^A-Za-zäëöü'`ꞌꞋ'‘’])z/Language either not given or an invalid string.N) ETElementr,rrsearchformatrrappend) r1r2rr3tokenr4Z word_formingr8Ztoken_elrrr add_word_elBs(       r@)eafcCsJxD|dD]6}x0|dd|jddD]}||q0Wq WdS)zRemove the tiers that use the "Included In" stereotype constraint Parameters: eaf: The root element of an EAF file Makes changes in place (returns nothing) z .//*[@CONSTRAINTS='Included_In']z.//*[@LINGUISTIC_TYPE_REF={}]'ZLINGUISTIC_TYPE_IDN)r/r=rremove)rAZ included_inZbad_tierrrr remove_included_inks$rD)typingrrxml.etree.ElementTreeetree ElementTreer:rrr;rr,r0r@rDrrrr s  )