B b[àb×ã@sêddlmZddlmmZddlZddlZe e dƒ¡Z e dde  de ddd¡e dd e  de dd d¡iZ e e d œd d „Zejd œdd„Zdd„Zejdœdd„Zee eje dœdd„Ze ejdœdd„ZdS)é)ÚListNzflibl_config.jsonÚ languagesÚ main_languagez([^Zvalid_charactersz])Úchild_language)ÚphraseÚlgcCsz|rrdd„dd„t| |¡DƒDƒ}g}d}x>|D]6}|dkrJ||7}q4|r`| d|¡d}| |¡q4W|SgSdS)z÷Tokenize an utterance based on specified word-forming characters. Parameters: phrase: the string to be tokenized lg: the language whose word-forming characters are to be used in tokenization Return list of tokens cSsg|] }|r|‘qS©r)Ú.0Újrrú-/home/sunny/Documents/lx/flexible/flexible.pyú sztokenize..cSsg|] }| ¡‘qSr)Ústrip)r Úirrr r sÚÚ.ú N)Ú word_formingÚsplitÚappend)rrZtokensZcollected_tokensZ punct_charsÚtokenrrr Útokenizes "  r)Úelc Cs(td|jdt|jƒd|jdt|ƒƒdS)z„Print the tag, attributes, text, and number of children of an ET.Element Parameters: el: Element to be printed zTag:z Attrs:z Text:z No. of Children:N)ÚprintÚtagÚstrÚattribÚtextÚlen)rrrr Ú print_el_info)srcCs¢ttddd ¡ƒd}t|ƒdd…}ddt|ƒ|}|dd …d |d d …d |d d …d |d d …d |d d…}tddd t|ƒ¡|S)zšGenerate FLEx guid based on offset defined in offset.txt Increments offset upon use. Return guid in format [0-f]{8}-([0-f]{4}-){3}[0-f]{12} z offset.txtÚr)ÚmodeééNÚ0é éú-é ééÚw)ÚintÚopenÚreadÚhexrÚwriter)Z global_offsetZ new_guid_numZ new_guid_strZnew_guidrrr Ú generate_guid1s Lr0)Úeaf_rootcCsdd„| d¡DƒS)zÈGet time IDs and values from an EAF file Parameters: eaf_root: is the root element of an EAF object parsed through ElementTree Return a dictionary of time ID and value pairs cSsi|]}|jd|jd“qS)Z TIME_VALUEZ TIME_SLOT_ID)r)r rrrr ú Hsztime_values..z .//TIME_SLOT)Úfindall)r1rrr Ú time_values@sr4)Ú tokenized_uttÚ phrase_elrc Cs„t d¡}t|}xb|D]Z}tjddtƒid}| |¡r@d}nd}tjd||dœd}||_| |¡| |¡qW| |¡d S) ašPopulate a phrase element with a tokenized utterance *tokenized_utt* is a list with tokens from an utterance *phrase_el* is the element that will be the parent to the words added (below the words el in the phrase_el will be the items w/ translations and notes) *lg* is the language whose word-forming characters are to be used in tokenization Makes changes in place (returns nothing) ÚwordsÚwordÚguid)rÚpunctZtxtÚitem)ÚtypeÚlangN)ÚETÚElementrr0Úsearchrr) r5r6rr7Zutterance_word_formingrr8r<Ztoken_elrrr Ú add_word_elLs    rA)Ú constraintÚeafcCsHxB| d |¡¡D].}x(| d |jd¡¡D]}| |¡q.WqWdS)zºRemove the tiers that use the "Included In" stereotype constraint Parameters: eaf: The root element of an EAF file Makes changes in place (returns nothing) z.//*[@CONSTRAINTS='{}']z.//*[@LINGUISTIC_TYPE_REF='{}']ZLINGUISTIC_TYPE_IDN)r3ÚformatrÚremove)rBrCZconstraint_tierZbad_tierrrr Úremove_constraintes rF)ÚtypingrÚxml.etree.ElementTreeÚetreeÚ ElementTreer>ÚjsonÚreÚloadr,ÚconfigÚcompilerrrr?rr0r4rArFrrrr Ús "&