How to Read a ROOT File via Amazon S3?

Amazon Simple Storage Service (Amazon S3) is a commercial service provided by Amazon that allows users, typically developers, to store data over the Amazon cloud. The service provides users a highly-scalable, reliable, and low-latency data storage infrastructure with a price calculated on a usage basis fashion dependent on the region the data is stored. The data which size range varies from bytes to terabytes can be uploaded via an unlimited number of files that are stored into buckets. Each file is associated with a bucket which is associated with a region. If you are an unexperienced S3 user follow the official Get Started to learn the basic tasks of Amazon S3 such as Create a Bucket and Upload Files.

How to access ROOT files via Amazon S3. Simple, just create a TAS3File object instead of a TFile object with a URL on the form of as3://server/bucket/file as file name. For example:

 
root [0] TFile *f = TFile::Open("as3://s3-eu-west-1.amazonaws.com/roots3/hsimple.root")
root [1] f->ls()
TAS3File**		as3://s3-eu-west-1.amazonaws.com/roots3/hsimple.root	Demo ROOT file with histograms
 TAS3File*		as3://s3-eu-west-1.amazonaws.com/roots3/hsimple.root	Demo ROOT file with histograms
  KEY: TH1F	hpx;1	This is the px distribution
  KEY: TH2F	hpxpy;1	py vs px
  KEY: TProfile	hprof;1	Profile of pz versus px
  KEY: TNtuple	ntuple;1	Demo ntuple
root [2] hpx->Draw()	

Since TAS3File inherits TWebFile which inherits from TFile, all TFile operations work as expected. However, since TWebFile is a readonly file, TAS3File will also be a readonly file. As TWebFile, TAS3File is ideally suited to read relatively small objects (like histograms or other data analysis results). Although possible, you don't want to analyse large TTree's via a TAS3File.

Before you start using Amazon S3 with ROOT a configuration step is required since communication between ROOT and Amazon S3 is conducted through HTTP requests that respect a RESTful API provided by Amazon S3.

Here follows a step-by-step recipe for authentication configuration between ROOT and Amazon S3:

1. Go to the main Amazon S3 website and login into your account.
2. Once you are logged in, click on Security Credentials on the left panel.
3. In the section Access Credentials, tab Access Keys retrieve both Access Key ID and Secret Access Key.
4. Open your bash profile file (usually ~/.bashrc or ~/.profile) and add the following lines:
   export S3_ACCESS_ID="your access id"
   export S3_ACCESS_KEY="your access secret key"
5. Restart your terminal for changes to be applied and confirm that those variables are set by typing printenv.

Those environment variables will be read by ROOT once you try to create a TAS3File to calculate a signature that is appended to each request sent to the Amazon S3 server. Although communication is safe since the actual signature is an encrypted string, the Secret Access Key that is used to calculate the signature can be retrieved by anyone who has access to your terminal. You should make sure that the system where you configure is safe and regularly renew your Secret Access Key.