Ingest Tool
Ingest Tool is a Maven project consisting of two modules,
- ingest-core: basic framework of ingest, including batch ingest based on Spring Batch, messaging based on Spring Integration
- ingest-web: integrate ingest-core with Spring Batch Admin
common tasks
- maven install/package: (is not recommended to manually test/package/install this project because it contains a list of properties, please refer to Jenkins job for ingest)
- deploy: bhladmin@bhl-int1: sudo ln -s {ingest_web_project}/dist/ingest-web.xxx.war /var/lib/tomcat6/webapp/ingest.war
- undeploy: bhladmin@bhl-int1: cd {ingest_web_project} ; mvn tomcat:undeploy -Dpath=ingest
location
dependencies
- Pre-ingest Tool
- Fedora Repository
modifying config files
- {ingest_core_classpath}: ingest.properties
- The original file is placed in {ingest_web_project}/src/main/resources/ingest.properties, all environment-dependent properties are masked as ${xxxx}
- Maven filters masked properties for each compile, e.g. mvn test -Dfedora.client.username = fedoraAdmin. (Incomplete properties will cause breakdown)
- Overriding properties with system properties in runtime is done by Spring, for more information, please refer to {ingest_core_project}/src/main/resources/META-INF/spring/env-context.xml
- {ingest_core_classpath}: logback.xml
- ${log.path} is filtered by Maven during build
- {ingest_classpath}: environment-test/prod.properties
- Platform-specific configuration files. These files contains the same list of parameters in Jenkins.
- Triggered by: mvn clean package -DskipTests -Denvironment=test (or -Denvironment=prod)
how to test if it's working
- Every build by Jenkins ensures passing all test cases.
- For testing of batch ingest, please go to http://bhl-int.nhm.ac.uk/ingest/jobs/batchIngestJob, fill job parameters with (of course the 'activation' step will fail because a000test does not exist)
- GUID=bhle:a000test
- URI=file:///{location_of_AIP_folder_path}
What if batch purge
- mvn exec:java -Dexec.mainClass="com.bhle.ingest.util.QueryAndPurge" -Dexec.args="{query}"
- {query}: e.g. *bhle*, bhle-10706-a000test*
- Purge all ingested SIP from Pre-ingest:
- mvn exec:java -Dexec.mainClass="com.bhle.ingest.util.QueryAndPurge" -Dexec.args="*10706*" (every object contains the GUID bankID 10706)
proxy information
production
- stop tomcat
cd /mnt/nfs/prod/opt/ingest/bin/
sh catalina.sh stop
- compile ingest.war for production environment
cd /mnt/nfs/dev/opt/ingest
mvn clean package -P prod -DskipTests
- clean ingest deployment files
cd /mnt/nfs/prod/opt/ingest/webapps/
rm -r ingest/
- start tomcat
cd /mnt/nfs/prod/opt/ingest/bin/
sh catalina.sh start