DWDS/Dialing Concordance Code

a collection of indexing and search tools for corpus linguists

Brought to you by: garabik, kzimmer, mukau, sokirko

Commit [r1113] Maximize Restore History

* fixed ddc_split in presence of legacy anonymized tokens (those beginning with -#)

- workaround (hack): all types with 0 occurrences in source corpus are assumed to by anonymized,
and are added to the output vocabulary of *every* partition -- this is a bit bloated but should work
* added ddc_index option --anonymize (-a): enable legacy anonymized tokens (disabled by default)
- fixes mantis #31586

Authored by: mukau 2018-10-23

Browse code at this revision

Parent: [r1112]

Child: [r1114]

changed	/ddc/trunk/Changes
changed	/ddc/trunk/configure.ac
changed	/ddc/trunk/doc/ddc_cfg.5
changed	/ddc/trunk/doc/ddc_files.5
changed	/ddc/trunk/doc/ddc_opt.5
changed	/ddc/trunk/doc/ddc_proto.5
changed	/ddc/trunk/doc/ddc_query.5
changed	/ddc/trunk/doc/ddc_server.opt.5
changed	/ddc/trunk/doc/ddc_tabs.5
changed	/ddc/trunk/src/ConcordLib/IndexSet.cpp
changed	/ddc/trunk/src/ConcordLib/IndexSetForLoadingStage.cpp
changed	/ddc/trunk/src/ConcordLib/IndexSetForLoadingStage.h
changed	/ddc/trunk/src/common/ddcConfigNoAuto.h
changed	/ddc/trunk/src/ddc_index/ddc_index.cpp
changed	/ddc/trunk/src/ddc_split/ddc_split.cpp

/ddc/trunk/Changes Diff Switch to side-by-side view

/ddc/trunk/configure.ac Diff Switch to side-by-side view

/ddc/trunk/doc/ddc_cfg.5 Diff Switch to side-by-side view

/ddc/trunk/doc/ddc_files.5 Diff Switch to side-by-side view

/ddc/trunk/doc/ddc_opt.5 Diff Switch to side-by-side view

/ddc/trunk/doc/ddc_proto.5 Diff Switch to side-by-side view

/ddc/trunk/doc/ddc_query.5 Diff Switch to side-by-side view

/ddc/trunk/doc/ddc_server.opt.5 Diff Switch to side-by-side view

/ddc/trunk/doc/ddc_tabs.5 Diff Switch to side-by-side view

/ddc/trunk/src/ConcordLib/IndexSet.cpp Diff Switch to side-by-side view

/ddc/trunk/src/ConcordLib/IndexSetForLoadingStage.cpp Diff Switch to side-by-side view

/ddc/trunk/src/ConcordLib/IndexSetForLoadingStage.h Diff Switch to side-by-side view

/ddc/trunk/src/common/ddcConfigNoAuto.h Diff Switch to side-by-side view

/ddc/trunk/src/ddc_index/ddc_index.cpp Diff Switch to side-by-side view

/ddc/trunk/src/ddc_split/ddc_split.cpp Diff Switch to side-by-side view