The Archives
-
Cloud
AWS and Hadoop
The New York Times used 100 Amazon EC2 instances and a Hadoop application to process 4TB of raw image TIFF data (stored in S3) into 1.1 million finished PDFs in the space of 24 hours at a computation cost of about $240 (not including bandwidth).
NY Times: Self-service, Prorated Super Computing Fun!The project was so easy, [...]
John Bullard
Loose Thinking, Tight Analysis
