You are viewing an old version of this page. View the current version.
Compare with Current |
View Page History
Working plan
Pig example
For now clone https://github.com/perdalum/warcbase
and checkout the pig-integration branch. Running the unit tests will run the above Pig Latin script on the provided test gzip'ed ARC file. The language distribution reported by Tika is:
Goals
Create UDFs for
- language detection using Tika
- identification using Tika
Labels:
None