B Yb!@sddlmZddlZddlmmZeedddZej dddZ d d Z ej d d d Z eeej dddZ ej dddZdS))ListN)phraselgcCsV|dkrtd}n|dkr(td}ntd|rRdddd||DDSgS) zTokenize an utterance based on specified word-forming characters. Parameters: phrase: the string to be tokenized lg: the language whose word-forming characters are to be used in tokenization Return list of tokens tcaz([^0-9A-Za-z\+#_])Zmtou([^A-Za-zäëöü'])z/Language either not given or an invalid string.cSsg|] }|r|qSr).0jrr//home/sunny/Documents/lx/flexible/flexible01.py sztokenize..cSsg|] }|qSr)strip)rirrr r s)recompile ValueErrorsplit)rrZnon_word_formingrrr tokenizes   r)elc Cs(td|jdt|jd|jdt|dS)zPrint the tag, attributes, text, and number of children of an ET.Element Parameters: el: Element to be printed zTag:z Attrs:z Text:z No. of Children:N)printtagstrattribtextlen)rrrr print_el_infosrcCsttdddd}t|dd}ddt||}|dd d |d d d |d d d |d d d |d d}tdddt||S)zGenerate FLEx guid based on offset defined in offset.txt Increments offset upon use. Return guid in format [0-f]{8}-([0-f]{4}-){3}[0-f]{12} z offset.txtr)modeN0 - w)intopenreadhexrwriter)Z global_offsetZ new_guid_numZ new_guid_strZnew_guidrrr generate_guid#s Lr+)eaf_rootcCsdd|dDS)zGet time IDs and values from an EAF file Parameters: eaf_root: is the root element of an EAF object parsed through ElementTree Return a dictionary of time ID and value pairs cSsi|]}|jd|jdqS)Z TIME_VALUEZ TIME_SLOT_ID)r)rr rrr >sztime_values..z .//TIME_SLOT)findall)r,rrr time_values2s r/) tokenized_utt phrase_elcCstd}xl|D]d}tjddtid}td}||rBd}nd}tjd|d d d}||_||||qW||d S) aDPopulate a phrase element with a tokenized utterance *tokenized_utt* is a list with tokens from an utterance *phrase_el* is the element that will be the parent to the words added (below the words el in the phrase_el will be the items w/ translations and notes) Makes changes in place (returns nothing) wordswordguid)rz([^0-9A-Za-z\+#_])punctZtxtitemztca-fonipa-x-etic)typelangN)ETElementr+r rsearchrappend)r0r1r2tokenr3Z word_formingr7Ztoken_elrrr add_word_elBs     r>)eafcCsJxD|dD]6}x0|dd|jddD]}||q0Wq WdS)zRemove the tiers that use the "Included In" stereotype constraint Parameters: eaf: The root element of an EAF file Makes changes in place (returns nothing) z .//*[@CONSTRAINTS='Included_In']z.//*[@LINGUISTIC_TYPE_REF={}]'ZLINGUISTIC_TYPE_IDN)r.formatrremove)r?Z included_inZbad_tierrrr remove_included_in[s$rC)typingrr xml.etree.ElementTreeetree ElementTreer9rrr:rr+r/r>rCrrrr s