Commit Graph
1347 Commits
Author SHA1 Message Date
Reece H. Dunn 1c8ed9c190 tokenizer.c: Support mac newlines. 2017-04-26 07:39:06 +01:00
Reece H. Dunn 7602c9ac18 tokenizer.c: Support linux newlines. 2017-04-26 07:34:57 +01:00
Reece H. Dunn bce44316bb Create a basic tokenizer API using a structure that mirrors the TtsTokenizer interface in the tts-dev-studio project. 2017-04-25 07:55:10 +01:00
Reece H. Dunn 3cc53d98f4 Add ucd.h to tokenizer.c to provide the definition of the ucd_category identifier for the emscripten build. 2017-04-24 18:49:09 +01:00
Reece H. Dunn 61d668c0cb ucd-tools: Inverted_Terminal_Punctuation eSpeakNG extended property support; use in clause_type_from_codepoint. 2017-04-22 19:09:49 +01:00
Reece H. Dunn 5c6bc0e556 Armenian emphasis mark (U+055B) is used for interjections, so treat it as an exclamation mark. 2017-04-22 18:54:21 +01:00
Reece H. Dunn bc13173ac4 ucd-tools: Punctuation_In_Word eSpeakNG extended property support; use in clause_type_from_codepoint. 2017-04-22 18:52:46 +01:00
Reece H. Dunn 1131d0924b ucd-tools: Optional_Space_After eSpeakNG extended property support; use in clause_type_from_codepoint. 2017-04-22 18:17:39 +01:00
Reece H. Dunn b932f3c493 ucd-tools: Extended_Dash eSpeakNG extended property support; use in clause_type_from_codepoint. 2017-04-22 15:55:44 +01:00
Reece H. Dunn 3100ca9d1b Use ucd_properties to implement clause_type_from_codepoint for supported types. 2017-04-22 15:20:19 +01:00
Reece H. Dunn be86091088 Use #defines for the ESPEAKNG_PROPERTY_ constants, so they can be used in things like switch expressions. 2017-04-22 14:33:51 +01:00
Reece H. Dunn af88820954 Merge commit 'd18d98b92ca42bf7b098d3d1fd873d9b0ee82e00' 2017-04-22 14:31:07 +01:00
Reece H. Dunn d18d98b92c Use #defines for the UCD_PROPERTY_ constants, so they can be used in things like switch expressions. 2017-04-22 14:28:41 +01:00
Reece H. Dunn 1c4ce3dcd3 tokenizer.c: create and use a clause_type_from_codepoint function, with tests. 2017-04-19 22:33:32 +01:00
Reece H. Dunn 92f703d98b Use defines instead of hard-coded numbers for more clause logic. 2017-04-18 07:14:45 +01:00
Reece H. Dunn 8749891069 Better specify the CLAUSE_ flags returned by ReadClause. 2017-04-17 22:04:45 +01:00
Reece H. Dunn 2e375362c4 ucd-tools: Paragraph_Separator eSpeakNG extended property support. 2017-04-17 20:20:42 +01:00
Reece H. Dunn 7fc4f5ece2 ucd-tools: Ellipsis eSpeakNG extended property support. 2017-04-17 17:35:26 +01:00
Reece H. Dunn 8d8c8b3b56 ucd-tools: Semi_Colon eSpeakNG extended property support. 2017-04-17 17:17:52 +01:00
Reece H. Dunn 9869ee051e ucd-tools: Colon eSpeakNG extended property support. 2017-04-17 16:18:57 +01:00
Reece H. Dunn 9ef03b8ac8 ucd-tools: Comma eSpeakNG extended property support. 2017-04-17 15:14:02 +01:00
Reece H. Dunn 5017153d62 ucd-tools: Exclamation_Mark eSpeakNG extended property support. 2017-04-17 13:47:50 +01:00
Reece H. Dunn 31d66fddb1 ucd-tools: Question_Mark eSpeakNG extended property support. 2017-04-17 12:50:04 +01:00
Reece H. Dunn 07cd2b12e1 ucd-tools: Full_Stop eSpeakNG extended property support. 2017-04-17 11:45:21 +01:00
Reece H. Dunn 9bbb16a74c Add ucd-tools proplist.c to the build. 2017-04-16 15:04:15 +01:00
Reece H. Dunn 2615242f05 Merge commit 'ff018e33df5763dc0ff1644c9479ee2351bee45e' 2017-04-16 15:02:08 +01:00
Reece H. Dunn ff018e33df Support compiling ucd.h with a strict C11 compiler. 2017-04-16 15:01:37 +01:00
Reece H. Dunn 691457e98d Add Prepended_Concatenation_Mark support from PropList.txt. 2017-04-16 14:36:27 +01:00
Reece H. Dunn 429b8f3629 Add Pattern_Syntax support from PropList.txt. 2017-04-16 14:17:43 +01:00
Reece H. Dunn d91a249c14 Add Pattern_White_Space support from PropList.txt. 2017-04-16 11:37:14 +01:00
Reece H. Dunn eecd7984c5 Add Variation_Selector support from PropList.txt. 2017-04-16 11:26:05 +01:00
Reece H. Dunn f7f228bc0c Add Sentence_Terminal support from PropList.txt. 2017-04-16 11:05:59 +01:00
Reece H. Dunn 628dfb8887 Add Other_ID_Continue support from PropList.txt. 2017-04-16 09:43:03 +01:00
Reece H. Dunn 90a3615c48 Add Other_ID_Start support from PropList.txt. 2017-04-16 09:34:31 +01:00
Reece H. Dunn a3f811aac1 Add Logical_Order_Exception support from PropList.txt. 2017-04-16 09:29:02 +01:00
Reece H. Dunn 66b2404ce3 Add Soft_Dotted support from PropList.txt. 2017-04-16 09:09:03 +01:00
Reece H. Dunn 423aa813aa Add Deprecated support from PropList.txt. 2017-04-15 12:25:00 +01:00
Reece H. Dunn e18564899c Add Other_Default_Ignorable_Code_Point support from PropList.txt. 2017-04-15 11:55:23 +01:00
Reece H. Dunn 2459a4fa8f Add Unified_Ideograph support from PropList.txt. 2017-04-14 20:58:31 +01:00
Reece H. Dunn 4ce8b61180 Extend ucd_property to 64-bits to allow all properties to be specified. 2017-04-14 20:12:02 +01:00
Reece H. Dunn 5b9300b042 Add Radical support from PropList.txt. 2017-04-14 17:57:18 +01:00
Reece H. Dunn e6c2c718b4 Add IDS_Trinary_Operator support from PropList.txt. 2017-04-14 17:40:57 +01:00
Reece H. Dunn c9af03ee8b Add IDS_Binary_Operator support from PropList.txt. 2017-04-14 17:13:28 +01:00
Reece H. Dunn fdf398ef84 Add Other_Grapheme_Extend support from PropList.txt. 2017-04-14 14:44:10 +01:00
Reece H. Dunn 7fc8ee0e09 Use ucd_properties to implement isalnum, isalpha, isupper and islower. 2017-04-14 13:47:35 +01:00
Reece H. Dunn 6553494c5c Remove noBreak from ucd_properties -- it is a DispositionType, not a PropList type. 2017-04-14 13:36:35 +01:00
Reece H. Dunn 9dc44bf0d4 Add Noncharacter_Code_Point support from PropList.txt. 2017-04-14 13:20:22 +01:00
Reece H. Dunn a184262d9b Add Other_Uppercase support from PropList.txt. 2017-04-14 13:08:56 +01:00
Reece H. Dunn 6a2a87e358 Add Other_Lowercase support from PropList.txt. 2017-04-14 13:00:02 +01:00
Reece H. Dunn 735a661232 Add Extender support from PropList.txt. 2017-04-14 12:03:34 +01:00