|Registration is open for the joint three-day SCAPE and OPF Hackathon: Hadoop-Driven Digital Preservation https://hadoop-driven-digital-preservation.eventbrite.co.uk.|
Monday 2 December - Wednesday 4 December
Austrian National Library (ONB)
The SCAPE Project and OPF are running a hackathon for developers and practitioners, focussing on Hadoop, an open source software framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. Hadoop is designed to scale out from single servers to thousands of machines.
Jimmy Lin from the University of Maryland will our guest speaker at the event. Jimmy has been working with Big Data and Hadoop for many years, with a focus on natural language processing and information retrieval. He spent an extended sabbatical at Twitter from 2010-2012 to work on large-scale analytics, on which he provides valuable insights in his 2013 HadoopsummitEU talk ‘Big Data Mining Infrastructure: The Twitter Experience” (http://www.youtube.com/watch?v=T5ZjSFnOxys). He has a book out on MapReduce (http://lintool.github.io/MapReduceAlgorithms/) and is currently working on a scalable rendering engine for web archives based on HBase.
We will be working with two digital preservation scenarios:
- Web Archiving: File Format Identification/Characterisation
- Digital Books: Quality Assurance, text mining (OCR Quality)
Alternatively, if you have something else you would like to work on using Hadoop, just let us know, we are keen to hear your ideas.
Practitioners and developers will work together in groups to address digital preservation challenges using Hadoop. Practitioners will take the role of issue champion, and will articulate their requirements to the developers and document them on the wiki. Developers will brainstorm ideas, and work on solutions to the issues. There will be regular check in points to get feedback and refine requirements. There will be prize for the best issue champion and development solution.
All participants will gain practical experience of using digital preservation tools in characterisation and quality assurance processes. We will provide step-by-step worksheets for those who are less familiar with using the command line, and our experts will be on hand to help you through them.
There will be plenty of opportunities for discussion. We have a session for sharing experiences implementing Hadoop at your organisation, research project reports and a break out space for lightening talks. We welcome suggestions for talks or discussions you would like to hear about.
The draft agenda is available here: Agenda - Hadoop Driven Digital Preservation
The event will be conducted in English.
Practitioners (digital librarians and archivists, digital curators, repository managers, or anyone responsible for managing digital collections) You will learn how Hadoop might fit your organisation, how to write requirements to guide development and gain some hands on experience using tools yourself and finding out how they work. To get the most out of this training course you will ideally have some knowledge or experience of digital preservation.
Developers of all experience can participate, from writing your first Hadoop jobs, to working on scalable solutions for issues identified in the scenarios.
Please register here: https://hadoop-driven-digital-preservation.eventbrite.co.uk.
OPF members are invited to attend free of charge. Please use the code issued by email to waive the fee.
Non-members are welcome to attend at a cost of €200. Morning and afternoon coffee breaks and lunch will be provided and are included in the registration fee.
Early bird rate register before 25 October to get 10% off.
Registration will close on Monday 25 November.
The event will be very hands-on. Please bring a laptop to fully participate in the event.
Please go to the Library entrance at Josefsplatz 1: signs will point you to the meeting rooms.
The nearest metro stations are: Karlsplatz U1, U2, U4, Herrengasse U3 and Stephansplatz U1, U3
Rooms for the event have been pre-reserved at the nearby Josefshof hotel. Rooms are for 95,- (single room). Participants who want to book a room here can either call or write an e-mail to the hotel and mention they are guests of the Austrian National Library (ONB) to get the reduced fee.
The Motel One hotel is also recommended by the ONB.
The event dinner will be held on Monday 2 December, 19:00 at Fromme Helene (http://www.frommehelene.at/)
(The dinner is self-paid. Please indicate if you would like to attend when registering)
If you have any questions about the event please email us: rebecca [at] openplanetsfoundation [dot] org Please mention 'Hadoop Hackathon' in the subject line to ensure a prompt reply.
For more information about the SCAPE project, please visit the website: http://www.scape-project.eu/