Parses PDF files of scientific articles based on naive bayes and sophisticated heuristics. The output is a XML file that contains the parsed data. Meta data is detected and marked as such.
The meta data contains the following elements:
- Title
- Authors
- Abstract
- Text
- Headlines
- Enumerations
- References (Literature)
In the first step, the text elements are divided into blocks (similar to paragraphs) and after that, predictions for each element are made.
The project contains three runnable classes that can work on given PDFs in batch mode via threading:
a) BatchHeuristic: A parser that uses defined heuristics and rules. Especially applicable for articles with a broad set of layouts (e.g. PeDocs, http://www.pedocs.de/).
b) BatchHybrid: A parser that uses machine learning (Naive Bayes) to find the correct element. Useful for e.g. ACL.
c) ModelGenerator: Generates a training model, used by BatchHybrid, from given PDF and XML file
Features
- Batch mode for fast execution
- Understands various article styles
- Includes a learning mechanism to adapt new styles