DWDS/Dialing Concordance Code

+ tagged ddc-2.1.21-rc1

v2.1.21-rc1:
* added suffix-query optimization via new index files CORPUS._suffix_TOKATTR
- new low-level evaluation method CStringIndexSet::QueryTokenListWithRightTruncation()
- new suffix-sorted index CIndexSetForQueryingStage::m_rIndex
- can result in drastically improved suffix query-times compared to old regex-based vocabulary scan:
+ *en : 24% faster (Amdahl S=1.24)
+ *chen : 169% faster (Amdahl S=3.69)
+ *ber : 432% faster (Amdahl S=5.32)
* added ddc_(index|union|split) code to create CORPUS._suffix_TOKATTR for all indices at build-time
- very small memory and disk footprint (<1% total index size, growth O(NTypes))
* suffix-indices are still optional: if not present, old regex-based vocabulary scan will be used

+ suffix-index support for ddc_(index|union|split)

+ suffix index: documented and cleaned up testing code
+ TODO: ddc_(index|union|split) integration

+ suffix-index tests

+ first stab at auxiliary suffix-index
- no ddc_index-support yet, test build with perl script scripts/ddc-make-suffix-index.perl
+ can be up to 7x faster for small-ish sets of matched types (e.g. *ber)
+ only ca. 20% improvement for large match-sets (e.g. *en)
- todo: optimize in-place suffix-comparison struct, avoid temporaries

+ tagged ddc-2.1.20

+ v2.1.20: bugfix for WITH

+ removed tag ddc-2.1.20

+ tagged ddc-2.1.20