Commit Graph
100 Commits
Author SHA1 Message Date
Samuel Thibault f919333021 fuzz: Also store timeout POC 2022-09-05 11:00:41 +02:00
Samuel Thibault 5b5d5e3c76 fuzz: cache build 2022-09-05 00:17:46 +02:00
Samuel Thibault 6f8aeb4ebc Typo 2022-09-05 00:02:03 +02:00
Samuel Thibault 6284272c31 Fix out-of-bound vowel_stress access
./src/espeak-ng -v ar --path=$PWD 5777

would access out of the vowel_stress array, because GetVowelStress, in the
phcode == phonSYLLABIC case, only increments count if stress is 0.  This
code seems fishy, and is not coherent with the loop in SetWordStress.  I
made it similar to the other case above, thus fixing the out-of-bound
access.

This is modifiying the stress in the tn test, I don't know whether this
is expected.
2022-08-28 17:13:32 +02:00
Samuel Thibault dccf295f20 Make distclean-local remove more espeak-ng-data files
If for whatever reason the build got broken, the recompilation of data
would fail because espeak-ng would be unable to re-open the current
files. One is then stuck until removing the files by hand. Let's make "make
distclean" clear these all to be sure to be able to restart from zero.
2022-08-28 16:41:10 +02:00
Samuel Thibault 8789e1e8e1 Increase buffer size for thousands processing
E.g. yue does not have many thousands separators, so the value may be very
large, possibly as much as maxint.
2022-08-27 01:32:56 +02:00
Samuel Thibault 3c220e5831 Fix calling utf8_in2
utf8_in2, when working in backward mode, is assuming that we are giving
the address of the last byte of the previous character (see all other
calls to utf8_in2). Otherwise, utf8_in2 returns the size of the current
multibyte character instead of that of the previous multibyte character.
2022-08-26 22:51:44 +02:00
Samuel Thibault 4412de57fd Fix number processing buffer overflow fix
This first reverts "Fix number_buf buffer overflow"
(commit ada93e2db0)

This for loop is apparently actually expected to to skip over NUL
characters.

Fixes #1302

Instead, this limits number processing to 32 digits, as break_numbers does
not support more and would provide bogus result with further digits.

Also fix the signedness of break_numbers so that the 32th bit
actually effectively works.
2022-08-26 22:22:01 +02:00
Samuel Thibault cbb2b32cb1 mb-fr*-en: Give separate names
Otherwise if e.g. only mbrola-fr4 is installed, when the application tries
to load voice en-french, it actually ends up trying to load mb-fr1-en.
2022-08-25 23:31:17 +02:00
Samuel Thibault c7cf819df7 Add .. test
Ref #1271
2022-08-04 10:58:47 +02:00
Samuel Thibault 21ea0ac960 Do not stop if we do not have a dictsource for the language 2022-07-21 01:29:22 +02:00
Samuel Thibault 76cb4ee1e3 fuzzing: Update voice list 2022-07-21 00:51:57 +02:00
Samuel Thibault ecec635b44 Cope with truncated utf8 input 2022-07-21 00:33:09 +02:00
Samuel Thibault 53348c5118 Avoid match1.del_fwd getting stale value 2022-07-21 00:25:45 +02:00
Samuel Thibault 60767a41c0 Avoid buffer underflow 2022-07-21 00:18:16 +02:00
Samuel Thibault ada93e2db0 Fix number_buf buffer overflow 2022-07-21 00:03:57 +02:00
Samuel Thibault dda0967b0d Make it clear where phoneme tests should be added 2022-07-17 22:19:02 +02:00
Samuel Thibault 482ee1da05 Add missing check for EOF 2022-07-02 02:43:23 +02:00
Samuel Thibault 0fa9737a00 CI: run 64bit before 32bit
So developers get the 64bit results first
2022-07-02 02:07:00 +02:00
Samuel Thibault f60a27a7cd Do not overflow ph_buf
With emojis, phoneme strings can be very long
2022-07-02 01:56:27 +02:00
Samuel Thibault 1da9ad10e8 Add tertiary translator
TranslateWord2 passes translator2 as tr to TranslateWord which may call
TranslateWord3, SpeakIndividualLetters, TranslateLetter, which was calling
SetTranslator2 again, thus freeing the very tr being used. Make that
latter use another translator.
2022-07-02 01:04:24 +02:00
Samuel Thibault 9decedb8c2 Fix missing checks for EOF 2022-06-30 12:41:18 +02:00
Samuel Thibault d8b5786330 Disable gradle in reprotest
It seems that reprotest is breaking the gradle build
2022-06-12 01:42:14 +02:00
Samuel Thibault 5a53282746 Merge pull request #1003 from ryandesign/patch-1
Enhance detection of gradle
2022-06-12 00:56:57 +02:00
Samuel Thibault dfc322743d MAKE_MEM_UNDEFINED: Fix warning 2022-06-12 00:26:17 +02:00
Samuel Thibault f4133364d1 Add more const qualifiers 2022-06-11 16:20:19 +02:00
Samuel Thibault 6f58f75693 Add missing phoneme table reset
When there are language switches, when we rewind to the start of the
phoneme list we have to reset the phoneme table back.

This avoids some branching that depends on undefined values, caught by
valgrind in the case e.g. of an emoji substitution that contains a
language switch.

Ref #874
2022-06-09 13:20:43 +02:00
Samuel Thibault 3b009398c9 Add missing stresslevel field initialization 2022-06-09 12:21:05 +02:00
Samuel Thibault e17f0764e3 valgrind CI: add missing debugging information 2022-06-09 12:06:10 +02:00
Samuel Thibault eed7db1575 Fix out-of-bound access
IsLetterGroup looks for heading \0, so the padding also needs to include
an \0.
2022-06-09 11:26:17 +02:00
Samuel Thibault 24e1eaa0cc Mark static arrays undefined when they are reused
So valgrind can potentially catch e.g. missing \0.
2022-06-09 09:48:43 +02:00
Samuel Thibault 7e608a3273 Make some buffers non-static
These don't actually need to survive function exit.
2022-06-09 09:46:19 +02:00
Samuel Thibault a660fc04cc Add more const qualifiers on static arrays
So we know they are just tables, get protected from overflows, and get shared between processes
2022-06-09 08:43:50 +02:00
Samuel Thibault 2d3a49b161 Make static arrays undefined on entry into LoadVoice
This would have caught #1200
2022-06-09 08:37:56 +02:00
Samuel Thibault afdf50bfd3 Add a note to avoid erroneous commits like e91d0a5
Fixes #1200
2022-06-09 08:12:45 +02:00
Samuel Thibault ad6efd3427 Merge pull request #1211 from seanbudd/fixAutoLangSwitching
Revert "code cleanup: don't add a \0 to voices_language in LoadVoice()"
2022-06-09 08:10:27 +02:00
Samuel Thibault a25849e4d5 configure.ac: Fix inline function test
fixes #1171
2022-04-24 18:46:27 +02:00
Samuel Thibault 9c72ff1d31 CI: allow manual trigger 2022-04-23 10:54:24 +02:00
Samuel Thibault 5087b05c52 Mark re-used "source" array as undefined
And fix the prefetches after \0.
2022-04-13 23:24:30 +02:00
Samuel Thibault 1228094127 Mark re-used phoneme_tab array as undefined
And make sure that the unfilled entries are NULL to catch any spurious
reference.

Also fix some missing phoneme table switch according to language
changes.
2022-04-13 23:24:13 +02:00
Samuel Thibault 199cbe4a1a Mark re-used phoneme_list as undefined
And fill the last phlist prepause and newword fields, otherwise they are
detected as undefined:

==483407== Conditional jump or move depends on uninitialised value(s)
==483407==    at 0x488E6AB: Generate (synthesize.c:1228)
==483407==    by 0x488FD94: SpeakNextClause (synthesize.c:1587)
==483407==    by 0x4887F56: Synthesize (speech.c:457)
==483407==    by 0x488884C: sync_espeak_Synth (speech.c:570)
==483407==    by 0x487B270: espeak_Synth (espeak_api.c:90)
==483407==    by 0x10ACA0: main (espeak-ng.c:691)
==483407==  Uninitialised value was created by a client request
==483407==    at 0x4884893: MakePhonemeList (phonemelist.c:155)
==483407==    by 0x4895712: TranslateClause (translate.c:2682)
==483407==    by 0x488FCCF: SpeakNextClause (synthesize.c:1569)
==483407==    by 0x4887F56: Synthesize (speech.c:457)
==483407==    by 0x488884C: sync_espeak_Synth (speech.c:570)
==483407==    by 0x487B270: espeak_Synth (espeak_api.c:90)
==483407==    by 0x10ACA0: main (espeak-ng.c:691)
==483407==
==483407== Conditional jump or move depends on uninitialised value(s)
==483407==    at 0x488E622: Generate (synthesize.c:1211)
==483407==    by 0x488FD94: SpeakNextClause (synthesize.c:1587)
==483407==    by 0x4887F56: Synthesize (speech.c:457)
==483407==    by 0x488884C: sync_espeak_Synth (speech.c:570)
==483407==    by 0x487B270: espeak_Synth (espeak_api.c:90)
==483407==    by 0x10ACA0: main (espeak-ng.c:691)
==483407==  Uninitialised value was created by a client request
==483407==    at 0x4884893: MakePhonemeList (phonemelist.c:155)
==483407==    by 0x4895712: TranslateClause (translate.c:2682)
==483407==    by 0x488FCCF: SpeakNextClause (synthesize.c:1569)
==483407==    by 0x4887F56: Synthesize (speech.c:457)
==483407==    by 0x488884C: sync_espeak_Synth (speech.c:570)
==483407==    by 0x487B270: espeak_Synth (espeak_api.c:90)
==483407==    by 0x10ACA0: main (espeak-ng.c:691)
==483407==

This is changing the ssml.test output, but with no audible difference,
so this is probably a real fix for it.
2022-04-13 23:24:04 +02:00
Samuel Thibault f1ce2ec423 Mark re-used ph_list2 array as undefined
And fix looking after end of array.
2022-04-13 23:23:51 +02:00
Samuel Thibault 7954418b88 Fix generated tarball
The tarball produced by make dist was missing a lot of files, so that
building from it would completely fail.
2022-04-13 09:18:12 +02:00
Samuel Thibault 15804e57a2 Constify tables
So they are protected from overflows, and shared between processes
2022-04-10 22:48:40 +02:00
Samuel Thibault 3ce135ac2b Mark re-used buffers as undefined
These are global arrays reused several times. When using them msan and
valgrind thus believe they are always initialized, which reduces their
capacity to detect uninitialized values. We can however explicitly tell them
when they are reused, and thus to be considered as uninitialized.
2022-04-10 22:42:51 +02:00
Samuel Thibault 43eb231d6c Merge pull request #1150 from sthibaul/CI-32bits
Add x86-32 CI
2022-04-10 22:41:31 +02:00
Samuel Thibault 36feb2dc5c Add x86-32 CI 2022-04-10 22:18:27 +02:00
Samuel Thibault a34d74ed43 Make envelope computation more robust
When pollint() returns 100.0, multiplying by 2.55 doesn't actually seem to
be getting 255 on i386. Multiplying by 255 and dividing by 100, however,
does (probably because float computation with small integer values are
guaranteed to have integer results).

Fixes #1151
2022-04-10 22:12:28 +02:00
Samuel Thibault 310e8fc768 Avoid using floating-point operation
This has varying behavior depending on toolchain and processor.

Fixes #1151
2022-04-10 20:48:43 +02:00
Samuel Thibault b3ba314ba7 CI: Add reprotest 2022-04-10 16:32:40 +02:00
Samuel Thibault e8253c6513 ssml: Use a larger buffer
When building espeak-ng in a directory that has a long path, 80 bytes to
store it is quite short.
2022-04-10 16:16:00 +02:00
Samuel Thibault 2878e91db0 Fix testsuite under various locales
When the current locale doesn't match the current voice, grep would be
surprised by the produced output and believe that this is not text, for
instance with LC_ALL=ru_RU.CP1251 we get:

  TEST      tests/language-replace.test
[...]
testing mk
grep: (standard input): binary file matches
2d1
< Translate 'пејзаж'

But we can give -a to grep so it always considers its input as text.
2022-04-10 15:40:09 +02:00
Samuel Thibault 90c035c406 Fix overflow while capping to signed 16bit 2022-04-08 01:59:31 +02:00
Samuel Thibault c91aa78363 LoadSoundFile: Fix char signedness 2022-04-08 01:24:50 +02:00
Samuel Thibault 2c524ec8df Fix loading wav files on big-endian architectures
Fixes #1152
2022-04-08 01:18:16 +02:00
Samuel Thibault 473cd9951a CI: avoid aborting the whole matrix on failure 2022-04-03 03:54:57 +02:00
Samuel Thibault c5696f5695 Add uploading config.log artifact on CI failure 2022-04-03 03:33:29 +02:00
Samuel Thibault 980b167e8f CI: Factorize configuration by using a build matrix 2022-04-03 03:00:11 +02:00
Samuel Thibault fc05678e41 CheckThousandsGroup: Fix checking heading character before thousand group 2022-04-02 21:13:55 +02:00
Samuel Thibault f352f1e43f CheckThousandsGroup: Avoid reading uninitialized data
For the case when word is smaller than 4 characters, we should not look at
the 3rd or 4th character before checking the previous ones, otherwise
we'd at best read uninitialized data, at worse non-existing data.
2022-04-01 20:18:00 +02:00
Samuel Thibault 075cac9b07 fr: Fix PR replacement
Ref #853
2022-04-01 19:36:00 +02:00
Samuel Thibault 02dd413a32 Add valgrind CI run
Now that all errors are fixed.
2022-04-01 00:22:26 +02:00
Samuel Thibault ee0a468f95 soundicon: Avoid calling remove with uninitialized name
fname_temp is not always getting filled.
2022-03-28 01:03:31 +02:00
Samuel Thibault f23265419d tests: Check value returned by espeak-ng
Otherwise we would miss errors produced on shutdown.
2022-03-28 00:55:41 +02:00
Samuel Thibault 0f9a474ed4 Make msan warnings fatal
Otherwise they will go unnoticed.
2022-03-28 00:47:33 +02:00
Samuel Thibault 127968a6f6 Add msan CI run
Now that all memory errors are fixed.

Ref #220
2022-03-25 22:39:33 +01:00
Samuel Thibault fe63f1b3e7 MatchRule: Do not overflow the text
Some rules test against character not being of a certain type. That may
match with the \0 end-of-text marker, and thus actually step over
it and let MatchRule continue with uninitialized data after it, leading
to potential random behavior.

This commits fixes it by making sure that we don't read past that \0.

This seems to be changing the pronunciation of "capitals" from k'apIt@Lz to
k'apIt,alz, I don't know why, I guess the rule for it was actually
bogus?
2022-03-25 22:49:41 +02:00
Samuel Thibault 41fe49ea34 asan: Re-enable new_delete_type_mismatch
Now that it is fixed by

dab5457620 "Fix deleting FrameManagerImpl*"
261954b4bd "Fix deleting SpeechWaveGenerator"
2022-03-25 19:22:23 +01:00
Samuel Thibault e1c8adfcb5 Add tsan CI run
Now that all tsan errors are fixed.

Ref #220
2022-03-23 19:06:37 +01:00
Samuel Thibault 2d4fcfaf17 Add lsan CI run
Now that all lsan errors are fixed.

Ref #220
2022-03-20 21:57:19 +01:00
Samuel Thibault 55b2e9b2a3 MatchRule: Prevent non-eating special characters from eating characters
Special characters such as N, S1, etc. are not actually eating
characters. Their treatment should thus *not* update pre_ptr and post_ptr,
otherwise those would underflow/overflow, e.g. in the case

@) s (_NS1  [z]

this would overflow. This for instance noticeable with the memory sanitizer:

ESPEAK_DATA_PATH=$PWD ./src/espeak-ng -qX "capitals"
Translate 'capitals'
  1	c        [k]

  1	a        [a]

  1	p        [p]

  1	i        [I]

  1	t        [t]

  1	a        [a]

  1	l        [l]
 20	l (C     [l]

==2837201==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7f7f4422744b in utf8_in2 /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:281:2
    #1 0x7f7f442281bc in utf8_in /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:332:9
    #2 0x7f7f440e0d31 in MatchRule /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/dictionary.c:1767:21
    #3 0x7f7f440d937f in TranslateRules /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/dictionary.c:2320:6
    #4 0x7f7f44230e5f in TranslateWord3 /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:733:15
    #5 0x7f7f44229844 in TranslateWord /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:1100:14
    #6 0x7f7f44256e50 in TranslateWord2 /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:1361:11
    #7 0x7f7f4424d6cc in TranslateClause /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:2623:17
    #8 0x7f7f44213359 in SpeakNextClause /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/synthesize.c:1569:2
    #9 0x7f7f441a9f56 in Synthesize /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/speech.c:457:2
    #10 0x7f7f441a9023 in sync_espeak_Synth /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/speech.c:570:29
    #11 0x7f7f441ad59f in espeak_ng_Synthesize /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/speech.c:678:10
    #12 0x7f7f4410b3f4 in espeak_Synth /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/espeak_api.c:90:32
    #13 0x4a8be3 in main /home/samy/brl/speech/espeak-ng-git/src/espeak-ng.c:691:3
    #14 0x7f7f43a2e7fc in __libc_start_main csu/../csu/libc-start.c:332:16
    #15 0x421449 in _start (/home/samy/ens/projet/1/speech/espeak-ng-git/src/.libs/espeak-ng+0x421449)

  Uninitialized value was created by an allocation of 'sbuf' in the stack frame of function 'TranslateClause'
    #0 0x7f7f4423a1f0 in TranslateClause /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:1941

While trying to match _NS1, MatchRule is overflowing the buffer.

It happens that this had not usually posed problem because rules usually
have these non-eating special characters last in the rule and thus it wasn't
mattering that post_ptr is pointing outside valid text.
2022-03-20 21:51:05 +01:00
Samuel Thibault caf8e650c2 MatchRule: Support multibyte doubled letters
When matching a double-letter whose encoding needs multiple bytes, to
properly skip the double-letter we have to skip these bytes, like other
rules.
2022-03-20 21:50:43 +01:00
Samuel Thibault 0f85657c7b MatchRule: Do not underflow the text
Some rules test against character not being of a certain type. That may
match with the \0 beginning-of-text marker, and thus actually step over
it and let MatchRule continue with uninitialized data before it, leading
to potential random behavior.

This commits fixes it by making sure that we don't read before that \0.
2022-03-20 02:56:37 +01:00
Samuel Thibault e87691ad3b Prevent LookupDict2 from overflowing wtab
LookupDict2 looks forward in the wtab array, it should still stop at its
end. Otherwise the memory sanitizer reports this:

testing en A. B C, D. E: F.
==65960==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7ff9d7ef0de8 in LookupDict2 /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/dictionary.c:2676:11
    #1 0x7ff9d7eec2ec in LookupDictList /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/dictionary.c:2899:10
    #2 0x7ff9d802860a in TranslateWord3 /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:588:12
    #3 0x7ff9d80249d4 in TranslateWord /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:1100:14
    #4 0x7ff9d8051fe0 in TranslateWord2 /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:1361:11
    #5 0x7ff9d804885c in TranslateClause /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:2623:17
    #6 0x7ff9d800e4e9 in SpeakNextClause /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/synthesize.c:1569:2
    #7 0x7ff9d7fa50e6 in Synthesize /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/speech.c:457:2
    #8 0x7ff9d7fa41b3 in sync_espeak_Synth /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/speech.c:570:29
    #9 0x7ff9d7fa872f in espeak_ng_Synthesize /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/speech.c:678:10
    #10 0x7ff9d7f06584 in espeak_Synth /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/espeak_api.c:90:32
    #11 0x4a8be3 in main /home/samy/brl/speech/espeak-ng-git/src/espeak-ng.c:691:3
    #12 0x7ff9d78297fc in __libc_start_main csu/../csu/libc-start.c:332:16
    #13 0x421449 in _start (/home/samy/ens/projet/1/speech/espeak-ng-git/src/.libs/espeak-ng+0x421449)

  Uninitialized value was created by an allocation of 'words' in the stack frame of function 'TranslateClause'
    #0 0x7ff9d8035380 in TranslateClause /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:1941
2022-03-20 02:24:13 +01:00
Samuel Thibault c8de29dec9 Properly compare strings
Strictly speaking, we are not supposed to use memcmp to compare strings
since we are not supposed to read beyond \0, which memcmp is supposed to
potentially do. Sanitizers would warn about it, and using strncmp happens to
provide the proper semantic while being not really slower, so better
just use them.
2022-03-20 00:35:27 +01:00
Samuel Thibault 4aeb78e788 LoadVoice: Fix phonemes_name initialization
phonemes_name is only initialized when V_LANGUAGE is met. This is not
necessarily the case, notably with

testing espeak_SetVoiceByName("!v/Annie") (language variant; intonation)
Cannot set intonation: language not set, or is invalid.
Uninitialized bytes in __interceptor_strcmp at offset 0 inside [0x7fff8a875e30, 1)
==4169902==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x4c6a49 in LookupPhonemeTable /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/synthdata.c:363:7
    #1 0x4c6a49 in SelectPhonemeTableName /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/synthdata.c:380:12
    #2 0x5098a9 in LoadVoice /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/voices.c:950:34
    #3 0x50edcf in espeak_ng_SetVoiceByName /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/voices.c:1585:7
    #4 0x4aad63 in espeak_SetVoiceByName /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/espeak_api.c:125:32
    #5 0x4a3fe1 in test_espeak_set_voice_by_name_language_variant_intonation_parameter /home/samy/brl/speech/espeak-ng-git/tests/api.c:356:2
    #6 0x4a3fe1 in main /home/samy/brl/speech/espeak-ng-git/tests/api.c:567:2
    #7 0x7f26e88cb7fc in __libc_start_main csu/../csu/libc-start.c:332:16
    #8 0x4213a9 in _start (/home/samy/ens/projet/1/speech/espeak-ng-git/tests/api.test+0x4213a9)

  Uninitialized value was created by an allocation of 'phonemes_name' in the stack frame of function 'LoadVoice'
    #0 0x504290 in LoadVoice /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/voices.c:519

so better catch it properly rather than relying on uninitialized data.
2022-03-20 00:31:58 +01:00
Samuel Thibault b457d6f58e Fix uninitialized fields
The memory sanitizer would complain:

==4157154==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7fc191d0a85b in TranslateWord3 /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:1065:7
    #1 0x7fc191d02916 in TranslateWord /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:1100:14
    #2 0x7fc191d1b324 in TranslateWord2 /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:1448:15
    #3 0x7fc191d14ebc in TranslateClause /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/translate.c:2623:17
    #4 0x7fc191cfbc9b in SpeakNextClause /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/synthesize.c:1569:2
    #5 0x7fc191cd52fc in Synthesize /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/speech.c:457:2
    #6 0x7fc191cd6d7c in sync_espeak_Synth /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/speech.c:570:29
    #7 0x7fc191cd6d7c in espeak_ng_Synthesize /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/speech.c:678:10
    #8 0x7fc191ca0340 in espeak_Synth /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/espeak_api.c:90:32
    #9 0x4a4381 in main /home/samy/brl/speech/espeak-ng-git/src/espeak-ng.c:691:3
    #10 0x7fc19168b7fc in __libc_start_main csu/../csu/libc-start.c:332:16
    #11 0x421449 in _start (/home/samy/ens/projet/1/speech/espeak-ng-git/src/.libs/espeak-ng+0x421449)

  Uninitialized value was created by a heap allocation
    #0 0x45000d in malloc (/home/samy/ens/projet/1/speech/espeak-ng-git/src/.libs/espeak-ng+0x45000d)
    #1 0x7fc191d1ca29 in NewTranslator /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/tr_languages.c:242:26
    #2 0x7fc191d1ca29 in SelectTranslator /home/samy/brl/speech/espeak-ng-git/src/libespeak-ng/tr_languages.c:482:7

(and similar for expect_verb_sn expect_noun, expect_past,
clause_upper_count, clause_lower_count)

Indeed TranslateWord3 doesn't always initialize these fields. Better
just initialize them directly from the Translator creation.
2022-03-20 00:27:44 +01:00
Samuel Thibault 9cee174426 sPlayer: fix accessing bw
The bw array is 0-based, not 1-based.
2022-03-19 10:55:06 +01:00
Samuel Thibault 261954b4bd Fix deleting SpeechWaveGenerator
Similarly to dab5457620 "Fix deleting FrameManagerImpl*", we need a
virtual destructor. clang was complaining about it:

src/speechPlayer/src/speechPlayer.cpp:52:2: warning: delete called on 'SpeechWaveGenerator' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]
        delete playerHandleInfo->waveGenerator;
        ^
2022-03-19 10:50:28 +01:00
Samuel Thibault f6b06e6db1 Fix asan usan CI merge 2022-03-16 23:43:26 +01:00
Samuel Thibault 15091d82be fifo: Protect setting my_terminate_is_required
We need to keep the mutex around setting it, to make sure that the
visibility of setting it to true doesn't get delayed, notably not after
the signal (which could entail that the signal becomes useless, leading
to a hang).
2022-03-16 00:22:58 +01:00
Samuel Thibault dab5457620 Fix deleting FrameManagerImpl*
delete playerHandleInfo->frameManager wasn't actually calling
FrameManagerImpl~, because playerHandleInfo->frameManager is only a
FrameManager, which didn't have a destructor, and nothing was saying that
it's actually a FrameManagerImpl behind. Adding a virtual destructor
fixes that.
2022-03-15 23:26:20 +01:00
Samuel Thibault 51af5a4155 Add KlattFiniSP
speechPlayer_terminate was never called without calling
speechPlayer_initialize right after, so there would always be a FrameManager
laying behind.
2022-03-15 21:27:44 +01:00
Samuel Thibault c790af66d0 Add asan CI run
Now that all asan errors are fixed.

Ref #220
2022-03-15 18:20:28 +01:00
Samuel Thibault 1ce5a1bb0b IsLetterGroup: Do not blindly walk back in the word
strlen(p) may be arbitrarily long, that would underflow the word, for
instance:

testing fr Latn
=================================================================
==3741805==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffd733c1329 at pc 0x7ff5ffbad2de bp 0x7ffd733bf310 sp 0x7ffd733bf308
READ of size 1 at 0x7ffd733c1329 thread T0
    #0 0x7ff5ffbad2dd in IsLetterGroup src/libespeak-ng/dictionary.c:714
    #1 0x7ff5ffbbe425 in MatchRule src/libespeak-ng/dictionary.c:1979
    #2 0x7ff5ffbc09e9 in TranslateRules src/libespeak-ng/dictionary.c:2301
    #3 0x7ff5ffc26656 in TranslateWord3 src/libespeak-ng/translate.c:733
    #4 0x7ff5ffc2a10b in TranslateWord src/libespeak-ng/translate.c:1100
    #5 0x7ff5ffc2bef2 in TranslateWord2 src/libespeak-ng/translate.c:1361
    #6 0x7ff5ffc374e2 in TranslateClause src/libespeak-ng/translate.c:2623
    #7 0x7ff5ffc1d010 in SpeakNextClause src/libespeak-ng/synthesize.c:1569
    #8 0x7ff5ffbfbd46 in Synthesize src/libespeak-ng/speech.c:492
    #9 0x7ff5ffbfd52a in sync_espeak_Synth src/libespeak-ng/speech.c:570
    #10 0x7ff5ffbfdd1f in espeak_ng_Synthesize src/libespeak-ng/speech.c:678
    #11 0x7ff5ffbc72fd in espeak_Synth src/libespeak-ng/espeak_api.c:90
    #12 0x5627511a3137 in main src/espeak-ng.c:691
    #13 0x7ff5fee557fc in __libc_start_main ../csu/libc-start.c:332
    #14 0x5627511a0569 in _start (/home/samy/ens/projet/1/speech/espeak-ng-git/src/.libs/espeak-ng+0x6569)

Address 0x7ffd733c1329 is located in stack of thread T0 at offset 1177 in frame
    #0 0x7ff5ffc2f760 in TranslateClause src/libespeak-ng/translate.c:1941

  This frame has 16 object(s):
    [48, 52) 'cc' (line 1944)
    [64, 68) 'source_index' (line 1945)
    [80, 84) 'prev_in' (line 1948)
    [96, 100) 'prev_out' (line 1949)
    [112, 116) 'next_in' (line 1952)
    [128, 132) 'char_inserted' (line 1954)
    [144, 148) 'word_flags' (line 1963)
    [160, 164) 'charix_top' (line 1975)
    [176, 180) 'tone' (line 1985)
    [192, 196) 'next2_in' (line 2294)
    [208, 212) 'c_temp' (line 2518)
    [224, 374) 'number_buf' (line 2522)
    [448, 1048) 'num_wtab' (line 2523)
    [1184, 1984) 'sbuf' (line 1982) <== Memory access at offset 1177 underflows this variable
    [2112, 3720) 'charix' (line 1977)
    [3856, 7456) 'words' (line 1978)

sbuf is however properly '\0'-header, so we can make IsLetterGroup
carefully walk back in the word and issue a mismatch if it walks back
too much.

Fixes #1108
2022-03-15 18:20:28 +01:00
Samuel Thibault 34a12c83b4 IsLetterGroup: Do not blindly walk back in the word
strlen(p) may be arbitrarily long, that would underflow the word, for
instance:

testing fr Latn
=================================================================
==3741805==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffd733c1329 at pc 0x7ff5ffbad2de bp 0x7ffd733bf310 sp 0x7ffd733bf308
READ of size 1 at 0x7ffd733c1329 thread T0
    #0 0x7ff5ffbad2dd in IsLetterGroup src/libespeak-ng/dictionary.c:714
    #1 0x7ff5ffbbe425 in MatchRule src/libespeak-ng/dictionary.c:1979
    #2 0x7ff5ffbc09e9 in TranslateRules src/libespeak-ng/dictionary.c:2301
    #3 0x7ff5ffc26656 in TranslateWord3 src/libespeak-ng/translate.c:733
    #4 0x7ff5ffc2a10b in TranslateWord src/libespeak-ng/translate.c:1100
    #5 0x7ff5ffc2bef2 in TranslateWord2 src/libespeak-ng/translate.c:1361
    #6 0x7ff5ffc374e2 in TranslateClause src/libespeak-ng/translate.c:2623
    #7 0x7ff5ffc1d010 in SpeakNextClause src/libespeak-ng/synthesize.c:1569
    #8 0x7ff5ffbfbd46 in Synthesize src/libespeak-ng/speech.c:492
    #9 0x7ff5ffbfd52a in sync_espeak_Synth src/libespeak-ng/speech.c:570
    #10 0x7ff5ffbfdd1f in espeak_ng_Synthesize src/libespeak-ng/speech.c:678
    #11 0x7ff5ffbc72fd in espeak_Synth src/libespeak-ng/espeak_api.c:90
    #12 0x5627511a3137 in main src/espeak-ng.c:691
    #13 0x7ff5fee557fc in __libc_start_main ../csu/libc-start.c:332
    #14 0x5627511a0569 in _start (/home/samy/ens/projet/1/speech/espeak-ng-git/src/.libs/espeak-ng+0x6569)

Address 0x7ffd733c1329 is located in stack of thread T0 at offset 1177 in frame
    #0 0x7ff5ffc2f760 in TranslateClause src/libespeak-ng/translate.c:1941

  This frame has 16 object(s):
    [48, 52) 'cc' (line 1944)
    [64, 68) 'source_index' (line 1945)
    [80, 84) 'prev_in' (line 1948)
    [96, 100) 'prev_out' (line 1949)
    [112, 116) 'next_in' (line 1952)
    [128, 132) 'char_inserted' (line 1954)
    [144, 148) 'word_flags' (line 1963)
    [160, 164) 'charix_top' (line 1975)
    [176, 180) 'tone' (line 1985)
    [192, 196) 'next2_in' (line 2294)
    [208, 212) 'c_temp' (line 2518)
    [224, 374) 'number_buf' (line 2522)
    [448, 1048) 'num_wtab' (line 2523)
    [1184, 1984) 'sbuf' (line 1982) <== Memory access at offset 1177 underflows this variable
    [2112, 3720) 'charix' (line 1977)
    [3856, 7456) 'words' (line 1978)

sbuf is however properly '\0'-header, so we can make IsLetterGroup
carefully walk back in the word and issue a mismatch if it walks back
too much.

Fixes #1108
2022-03-15 18:17:42 +01:00
Samuel Thibault 7f1222c6ad Properly maintain margin in ph_list2
TranslateWord2 uses phonemes in ph_list2. Apart from the breakable loops, it
may statically require up to 7 phonemes. Then TranslateClause always
uses 2 phonemes. We thus have to keep these margins along the loops to
avoid any overflow.

Fixes #1073 #1095
2022-03-15 15:47:09 +01:00
Samuel Thibault 26a675543c Properly maintain margin in ph_list2
TranslateWord2 uses phonemes in ph_list2. Apart from the breakable loops, it
may statically require up to 7 phonemes. Then TranslateClause always
uses 2 phonemes. We thus have to keep these margins along the loops to
avoid any overflow.

Fixes #1073
2022-03-15 04:08:51 +01:00
Samuel Thibault 7b9c99cec7 Add more heading margin in TranslateClause
Valgrind reports:

==3642987== Conditional jump or move depends on uninitialised value(s)
==3642987==    at 0x491F268: TranslateNumber_1 (numbers.c:1785)
==3642987==    by 0x4923C35: TranslateNumber (numbers.c:2080)
==3642987==    by 0x49556DC: TranslateWord3 (translate.c:644)
==3642987==    by 0x4957FCE: TranslateWord (translate.c:1100)
==3642987==    by 0x4959344: TranslateWord2 (translate.c:1361)
==3642987==    by 0x496116E: TranslateClause (translate.c:2613)
==3642987==    by 0x494FF7A: SpeakNextClause (synthesize.c:1569)
==3642987==    by 0x4939B9D: Synthesize (speech.c:457)
==3642987==    by 0x493AE6A: sync_espeak_Synth (speech.c:570)
==3642987==    by 0x493B286: espeak_ng_Synthesize (speech.c:678)
==3642987==    by 0x4916925: espeak_Synth (espeak_api.c:90)
==3642987==    by 0x10CF5D: main (espeak-ng.c:691)
==3642987==  Uninitialised value was created by a stack allocation
==3642987==    at 0x495BD9F: TranslateClause (translate.c:1941)

Indeed, TranslateNumber_1 looks back up to three bytes before, with
IsDigit09(word[-3])), so we have to increase the heading margin to three
spaces.
2022-03-15 04:08:51 +01:00
Samuel Thibault cd33c4042a MatchRule: Do not go back before the pre-ptr
pre_ptr is already one byte before the current letter, so we do not want
to subtract 1 again. Otherwise this would for instance underflow word_iz
of addPluralSuffixes.
2022-03-15 04:08:51 +01:00
Samuel Thibault 005a590786 addPluralSuffixes: Use space as heading separator
Otherwise asan reports this during make check:

testing en ibm mit ibms mits IBM MIT APH CES ITX IBMs MIT's APHs CES's ITXs
==3733154==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffe420233ef at pc 0x7f2e8a30aef1 bp 0x7ffe42022c80 sp 0x7ffe42022c78
READ of size 1 at 0x7ffe420233ef thread T0
    #0 0x7f2e8a30aef0 in utf8_in2 src/libespeak-ng/translate.c:281
    #1 0x7f2e8a2a6db1 in MatchRule src/libespeak-ng/dictionary.c:2058
    #2 0x7f2e8a2a89e9 in TranslateRules src/libespeak-ng/dictionary.c:2301
    #3 0x7f2e8a30cc77 in addPluralSuffixes src/libespeak-ng/translate.c:393
    #4 0x7f2e8a30e2c9 in TranslateWord3 src/libespeak-ng/translate.c:684
    #5 0x7f2e8a31210b in TranslateWord src/libespeak-ng/translate.c:1100
    #6 0x7f2e8a313ef2 in TranslateWord2 src/libespeak-ng/translate.c:1361
    #7 0x7f2e8a31f4e2 in TranslateClause src/libespeak-ng/translate.c:2623
    #8 0x7f2e8a305010 in SpeakNextClause src/libespeak-ng/synthesize.c:1569
    #9 0x7f2e8a2e390e in Synthesize src/libespeak-ng/speech.c:457
    #10 0x7f2e8a2e552a in sync_espeak_Synth src/libespeak-ng/speech.c:570
    #11 0x7f2e8a2e5d1f in espeak_ng_Synthesize src/libespeak-ng/speech.c:678
    #12 0x7f2e8a2af2fd in espeak_Synth src/libespeak-ng/espeak_api.c:90
    #13 0x5618104c9137 in main src/espeak-ng.c:691
    #14 0x7f2e8953d7fc in __libc_start_main ../csu/libc-start.c:332
    #15 0x5618104c6569 in _start (/home/samy/ens/projet/1/speech/espeak-ng-git/src/.libs/espeak-ng+0x6569)

Address 0x7ffe420233ef is located in stack of thread T0 at offset 47 in frame
    #0 0x7f2e8a30cb3b in addPluralSuffixes src/libespeak-ng/translate.c:380

  This frame has 3 object(s):
    [32, 36) 'word_zz' (line 381)
    [48, 52) 'word_iz' (line 382) <== Memory access at offset 47 underflows this variable
    [64, 68) 'word_ss' (line 383)

and indeed, RULE_NOVOWELS keeps looking back until it finds a spacing
character, so we have to provide it with one.
2022-03-15 03:40:52 +01:00
Samuel Thibault 0119efec28 MatchRule: Do not go back before the pre-ptr
pre_ptr is already one byte before the current letter, so we do not want
to subtract 1 again. Otherwise this would for instance underflow word_iz
of addPluralSuffixes.
2022-03-15 03:39:12 +01:00
Samuel Thibault 156b0fd45f Add usan CI run
Now that all usan errors are fixed.

Ref #220
2022-03-15 03:17:38 +01:00
Samuel Thibault 06221157e0 Merge branch 'n_ph_list2' into CI-usan 2022-03-15 03:11:25 +01:00
Samuel Thibault b81aac83a6 Merge branch 'translatenumber' into CI-usan 2022-03-15 03:11:19 +01:00
Samuel Thibault a932e86b11 Merge branch 'uninitialized' into CI-usan 2022-03-15 03:11:11 +01:00
Samuel Thibault eb9c7347c2 Merge branch 'integer-shift' into CI-usan 2022-03-15 03:10:55 +01:00
Samuel Thibault 910311ede1 Merge branch 'no-negative-index' into CI-usan 2022-03-15 03:10:46 +01:00
Samuel Thibault 165be6cc3a Merge branch 'wrapv' into CI-usan 2022-03-15 03:10:37 +01:00
Samuel Thibault 4dd1e04d8a Make polint use 0-based indexing
This avoids having to make LoadEnvelope2 pass &env_x[-1], which is very
strongly frowned upon by sanitizers such as usan.
2022-03-15 03:03:41 +01:00