Commit Graph
67 Commits
Author SHA1 Message Date
Reece H. Dunn 3189d194f1 Testing UTS 51 (5.0-12) ED-3. emoji character 2017-05-20 15:52:58 +01:00
Reece H. Dunn 27a08c5fea Test ReadClause on single character emoji sequences. 2017-05-20 10:23:59 +01:00
Reece H. Dunn 5aa47e01ea Simplify the readclause test names.
This is to support additional logic (i.e. testing the ReadClause
function) in the test functions.
2017-05-19 20:40:15 +01:00
Reece H. Dunn 37bae427c4 Remove the current voice tests.
These tests are limited in scope and should really be part of a
more comprehensive API tests.
2017-05-19 20:11:10 +01:00
Reece H. Dunn 48ca2239bb Fix non-Latin character languages falling back to English when reading Latin characters. 2017-05-12 07:35:27 +01:00
Reece H. Dunn d674529b9c tests/languages.test: Rename the test function to test_lang, to avoid a conflict with the test command. 2017-05-12 07:31:48 +01:00
Reece H. Dunn 5030ff95cc automake: don't make the tests print 'done', have the make rule print 'PASSED' instead. 2017-05-09 08:01:21 +01:00
Reece H. Dunn 8821c9e361 Split out the readclause tests so the tokenizer tests can use public-only APIs. 2017-05-09 07:44:42 +01:00
Reece H. Dunn d72557aed2 Add simple tests for testing voice selection by name. 2017-05-08 07:56:49 +01:00
Reece H. Dunn 6ec3e85007 Add language.tests for de, en and jp to test the phoneme generation. 2017-05-06 11:00:18 +01:00
Reece H. Dunn dd90d3812d tokenizer.c: Support general symbol tokens. 2017-05-03 08:08:40 +01:00
Reece H. Dunn 786575c6ed tokenizer.c: Support general punctuation tokens. 2017-05-03 07:56:53 +01:00
Reece H. Dunn 683579f403 Make the tokenizer.h API public. 2017-05-02 18:52:06 +01:00
Reece H. Dunn 9af96da469 Make the encoding.h API public. 2017-05-02 18:40:16 +01:00
Reece H. Dunn 55bfbb4754 tokenizer.c: Support ellipsis tokens. 2017-05-02 07:28:29 +01:00
Reece H. Dunn b847df63b5 tokenizer.c: Support semicolon tokens. 2017-05-01 19:38:37 +01:00
Reece H. Dunn af7e8fc5a3 tokenizer.c: Support colon tokens. 2017-05-01 19:22:48 +01:00
Reece H. Dunn 7560070dcd tokenizer.c: Support comma tokens. 2017-05-01 19:14:47 +01:00
Reece H. Dunn c9199cfacb tokenizer.c: Support exclamation mark tokens. 2017-05-01 19:00:40 +01:00
Reece H. Dunn 128ceaff6a tokenizer.c: Support question mark tokens. 2017-05-01 17:50:54 +01:00
Reece H. Dunn 8f62e18324 tokenizer.c: Support full stop tokens. 2017-05-01 17:45:55 +01:00
Reece H. Dunn d50f3f2fa5 tokenizer.c: Support word tokens. 2017-05-01 09:37:23 +01:00
Reece H. Dunn a902f451d8 tests/tokenizer.test: Support printing the tokens from a provided file, making it easy to investigate tokenizer issues. 2017-04-29 10:13:16 +01:00
Reece H. Dunn d093513b65 tokenizer.c: Add an options parameter to the tokenizer_reset API. 2017-04-28 07:48:55 +01:00
Reece H. Dunn c41ac642fa tokenizer.c: Tokenise Zp codepoints as paragraphs. 2017-04-28 07:44:22 +01:00
Reece H. Dunn f3ea6f68f3 tokenizer.c: Tokenise U+000B [VERTICAL TAB (VT)] as whitespace, not as newlines. 2017-04-28 07:37:00 +01:00
Reece H. Dunn fc7a4e6701 tokenizer.c: Recognise U+000C [FORM FEED (FF)] as a newline codepoint. 2017-04-28 07:31:05 +01:00
Reece H. Dunn d2d718d700 tokenizer.c: Tokenize line separator codepoints as newline tokens. 2017-04-28 07:16:38 +01:00
Reece H. Dunn bf45e7ce36 tokenizer.c: Recognise U+0085 [NEW LINE (NEL)] as a newline codepoint. 2017-04-28 07:08:02 +01:00
Reece H. Dunn df6ca7a22c tokenizer.c: Support whitespace tokens. 2017-04-27 07:59:29 +01:00
Reece H. Dunn 8f0dae6a38 tokenizer.c: Support windows newlines. 2017-04-26 07:52:33 +01:00
Reece H. Dunn b897ff5aa8 encoding.c: Support calling peekc past the end of the buffer. This makes calling peekc easier. 2017-04-26 07:48:34 +01:00
Reece H. Dunn 3f692f498b encoding.c: Implement a peekc API. 2017-04-26 07:45:21 +01:00
Reece H. Dunn 1c8ed9c190 tokenizer.c: Support mac newlines. 2017-04-26 07:39:06 +01:00
Reece H. Dunn 7602c9ac18 tokenizer.c: Support linux newlines. 2017-04-26 07:34:57 +01:00
Reece H. Dunn bce44316bb Create a basic tokenizer API using a structure that mirrors the TtsTokenizer interface in the tts-dev-studio project. 2017-04-25 07:55:10 +01:00
Reece H. Dunn 5c6bc0e556 Armenian emphasis mark (U+055B) is used for interjections, so treat it as an exclamation mark. 2017-04-22 18:54:21 +01:00
Reece H. Dunn 1c4ce3dcd3 tokenizer.c: create and use a clause_type_from_codepoint function, with tests. 2017-04-19 22:33:32 +01:00
Reece H. Dunn 9dabf64680 encoding.c: Support determining the string length for length < 0. 2017-04-01 13:23:09 +01:00
Reece H. Dunn b5ed1f28a5 encoding.c: Don't crash if NULL is passed as the string to the decode APIs. 2017-04-01 13:00:00 +01:00
Reece H. Dunn d167d5649b encoding.c: Implement support for the auto-detected character set (utf-8 + codepoint-encoding). 2017-03-31 22:30:27 +01:00
Reece H. Dunn 6a0b5e4ae1 encoding.c: Support using wchar_t strings with the text decoder API. 2017-03-30 22:52:38 +01:00
Reece H. Dunn b74f756f00 encoding.c: Support the ISO-10646-UCS-2 encoding. 2017-03-30 22:33:22 +01:00
Reece H. Dunn fa5d31a8af encoding.c: Support the UTF-8 encoding. 2017-03-30 21:17:56 +01:00
Reece H. Dunn 2499610433 encoding.c: Support the ISCII encoding. 2017-03-29 08:01:24 +01:00
Reece H. Dunn 39f3ea54cf encoding.c: Support the KOI8-R encoding. 2017-03-29 07:49:48 +01:00
Reece H. Dunn b8a1006dd8 encoding.c: Support the ISO 8859-16 encoding. 2017-03-29 07:42:00 +01:00
Reece H. Dunn 166e815723 encoding.c: Support the ISO 8859-15 encoding. 2017-03-29 07:36:42 +01:00
Reece H. Dunn 91e054ec7c encoding.c: Fix the ISO 8859 encoding names with date suffices. 2017-03-29 07:27:30 +01:00
Reece H. Dunn 0235c42652 encoding.c: Support the ISO 8859-14 encoding. 2017-03-29 07:19:16 +01:00