DWDS/Dialing Concordance Code

v2.1.24-rc1
* use new ddcCorpusList::rfind() for CConcIndexator::LoadMaskedFiles()
- string-based file-masking only masks the LAST occurrence of filename key (rather than the FIRST)
- backwards-compatible for old-style unique-filename corpora
- should also Do The Right Thing for "replaced" files snuck in by dstar-ddc-update.perl
- more precise control of file-masking possible with binary CORPUS._masked_ids

+ fixes for clean compilation under gcc v7.3.0 (e.g. ubuntu 18.04.1 LTS)

* new typedef CFileNo = DWORD
* new typedef CMaskedFileSet = set<CFileNo> for CConcIndexator::m_MaskedFiles
* propagate CConcIndexator::m_MaskedFiles correctly through ddc_union, ddc_split using new SaveMaskedFileIds() method

* propagate CORPUS._masked_ids through ddc_union

* added support for CORPUS._masked_ids : binary variant of CORPUS._masked
- quick and dirty implementation uses existing resident set<size_t> CConcIndexator::m_MaskedFiles
* added support for CConcIndexator::m_MaskedFiles in CConcIndexator::DumpIndex() (ddc_dump)

v2.1.23: bug-fix for suffix search (leftmost suffix character wasn't getting tested correctly;
search (*aufzug) in ND found "Aufzug"; culprit was bogus comparison ("ufzug" < "aufzug")

v2.1.22: doc updates

v2.1.22: bug-fixes for new prefix- and suffix-search introduced in v2.1.21 (mantis #33072 [bams|zeit])
* speed improvements too
- prefix searches are 7-26% faster than ddc-v2.1.18
- suffix search performance gains from v2.1.21 remain; especially for highly specific suffixes:
+ *haus : ca. 32x faster (S=32.07)
+ *erde : ca. 74x faster (S=74.49)

+ ddc-2.1.21 release

v2.1.21-rc1:
* added suffix-query optimization via new index files CORPUS._suffix_TOKATTR
- new low-level evaluation method CStringIndexSet::QueryTokenListWithRightTruncation()
- new suffix-sorted index CIndexSetForQueryingStage::m_rIndex
- can result in drastically improved suffix query-times compared to old regex-based vocabulary scan:
+ *en : 24% faster (Amdahl S=1.24)
+ *chen : 169% faster (Amdahl S=3.69)
+ *ber : 432% faster (Amdahl S=5.32)
* added ddc_(index|union|split) code to create CORPUS._suffix_TOKATTR for all indices at build-time
- very small memory and disk footprint (<1% total index size, growth O(NTypes))
* suffix-indices are still optional: if not present, old regex-based vocabulary scan will be used