TRevolution.js

Submitted by axel on Wed, 09/11/2011 - 22:04

Hi,

One of ROOT's traditional features: you can use it on any platform. That was especially true in the past: Linux, Windows, MacOS, Solaris, AIX, HPUX - you name it: ROOT was there. But now we have a different environment: devices are getting smaller, and next to good old Linux and Windows in new cloths (Android, Windows Mobile) we have new, dedicated mobile OSes like iOS.

Is that mobile world relevant to ROOT? Yes, because ROOT consists of two parts: the number crunching and storage - that didn't change too much - and the interactive part: experiments use ROOT histograms for online quality control, PROOF and any other well-designed compute "farming" solution reports back with live histograms.

So what would it take? We could implement an iOS, an Android, and a Windows Mobile port of ROOT. Timur and Fons have done the port to iOS, a student down my corridor, Jakob Blomer, once showed that a port to Android also works. The main challenges are the GUI / windowing systems, as well as the distribution of the ROOT port onto these devices. The Android port is incomplete on the graphics part, the iOS one on the distribution part. (Just because you own an iWhatever doesn't mean Apple allows you to do with it what you want. There's a good reason for it, though: Apple knows that that's better for you.) And both iOS and Android ports are too heavy for the most common use case: looking at histograms stored in ROOT files on a web server.

The obvious solution sounded demanding: implement ROOT's I/O in JavaScript, and pipe the data into one out of many possible JavaScript visualization libraries. While for iOS and Android we were able to use ROOT's native C++ core including I/O, all of that would have to be re-implemented for JavaScript. And ROOT's I/O is famous for its simplicity of implementation and its massive documentation. Err, wait. ROOT's I/O is famous for being able to cope with all C++ constructs and doing that incredibly sturdy and fast. Right, that's what it was.

Bertrand took up that challenge and implemented a pre-alpha prototype that already gives a good feeling just how powerful this approach is: grab the HTML file, the CSS file and the JavaScript file. Edit the HTML file to point to your ROOT file(s), put everything on a web server, and you can immediately browse the histograms the ROOT file contains, live. No ROOT installation, all JavaScript. No heavy lifting, only those bits are transferred that are needed: why download the 2GB TTree of you want to check out the histogram next to it?

I was super-impressed that Bertrand managed all that within a few weeks. That it's possible at all, actually. And how useful the result is, even if Bertrand claims it's far from being ready for public use (you have been warned! Especially the visualization part is now under active development).

I think his work will rapidly change the way the experiments do their online monitoring, the way people run analyses - check your intermediary results, live! - and possibly also the way you, our uses, see ROOT: not stuck in the dark origins of C++, but creatively making use of new technologies like your cell phone. We hope you'll like it - I do.

Cheers,
Axel

Edit 2013-01-15: updated CSS and JavaScript links.

Amazing! Any updates since

Submitted by Andrey (not verified) on Wed, 14/11/2012 - 21:53.
Amazing! Any updates since then? Unfortunately CSS & JS files are not available for downloading (Error 404).

Hi,The latest version of

Submitted by bertrand (not verified) on Thu, 15/11/2012 - 16:07.
Hi,

The latest version of the code is available at http://root.cern.ch/js, and you can also check it out of svn, from http://root.cern.ch/svn/root/trunk/js/JSRootIO

Cheers, Bertrand.



plot branch content

Submitted by Ilija Vukotic (not verified) on Tue, 14/02/2012 - 18:23.
Hi, thanks for very nice tool. I was wandering if it is possible to plot content of a branch or just histograms/graphs stored in the file? Is anybody thinking about porting a bit to node.js? -- Ilija

Re: Branches, node.js

Submitted by axel on Wed, 15/02/2012 - 12:29.

Hi Ilija!

Plotting branches means that javascript has to do the histogramming. And once you do that you probably want to have expressions. And once you do that you probably also want to have something similar to TTree::Draw()... While all of this is possible Bertrand focusses for now on displaying histograms and graphs (and, big news: he's currently working on displaying TCanvases, with all histograms / graphs that it contains!)

Let's see how it is used and where development will go from here!

Regarding node.js: nothing planned; and I'm afraid I don't see a realistic application for that, probably because of my limited imagination... What's you idea here?

Cheers, Axel.

Looks prety nice

Submitted by Isidro (not verified) on Fri, 10/02/2012 - 11:32.
Hi, This tool looks really nice and promising. I am looking forward to seeing the first full release. Hope it includes the statistics box with the histograms.

Hi. Has there been any

Submitted by Adam (not verified) on Sun, 29/01/2012 - 18:01.
Hi. Has there been any updates on this great tool? thanks, Adam

RE: Has there been any

Submitted by bellenot on Tue, 31/01/2012 - 10:36.

Hi Adam,

Thanks for your interest in this project! I'm still working on it... And the latest "public" demo version is still available there. It will be updated from time to time, to reflect latest changes.

Cheers, Bertrand.

The best ROOT feature ever

Submitted by Ricardo Neves (not verified) on Mon, 12/12/2011 - 01:10.
Hi everyone. In first place, let me thank you for this development. I can't count the number of uses this will have! The cherry on top of the cake would be a php api so one could generate histograms dynamically on the server side before displaying it... Is there anything similar already available?

Re: Best feature ever

Submitted by axel on Mon, 12/12/2011 - 17:52.

Hi Ricardo,

Thanks for your enthusiastic comment! I like cherries and cake: can we meet to discuss what exactly you envision? Are you at CERN or will you be here soon? Let me know (axel@cern.ch)!

Cheers, Axel

Slightly scary, but very

Submitted by Peter (not verified) on Sat, 26/11/2011 - 21:29.
Slightly scary, but very cool. We definitely have an immediate use case for this in ATLAS data quality monitoring, as I would imagine offloading histogram rendering to the clients would drastically reduce the load on our servers and the number of ROOT critical sections that we put locks around (as well as allowing dynamic zooming etc. as in the demo).

Re: Scary but useful

Submitted by axel on Sat, 26/11/2011 - 23:18.

Hi Peter,

Thanks for your comment! That dynamic interaction and the low overhead for publishing histograms were two of the main reasons to do this. Yes, your servers will just send out bytes, and apache is pretty good at that.

Note that we are just entering this new world - we have no experience with publishing this code / API properly, to make it usable by many possible applications while keeping the most common case (displaying histograms of a root file) as simple as possible. Luckily we have a wizard of Javascript right down the corridor; I hope he can help us with that.

Nevertheless, I'd really appreciate if you could contact Bertrand.Bellenot@cern and discuss how you'd like to use his code; this would definitely help with defining the interfaces.

Cheers, Axel.

Nice!

Submitted by Gordon (not verified) on Sat, 26/11/2011 - 12:57.
Wow. This just worked on everything I tried, which included, btw, Windows Phone 7.1 and IE9. And it was fast. Even on the mobile device rendering occured in well under a second, and my phone is over a year old.

Re: Everywhere

Submitted by axel on Sat, 26/11/2011 - 22:59.

Hi Gordon,

Thanks for trying it out! The main concerns people had in the beginning (and I'm rephrasing "why people thought the idea was nuts" here) were bandwidth and latency for getting the data, as well as CPU power for unzipping and unboxing, i.e. converting raw bytes into structured data members. It's great to see it works for accessing a 1GB ROOT file sitting in Asia from CERN. Or for rendering histograms on a Windows phone - any phone.

Now let's see how we can use it and what the next step could be!

Cheers, Axel

Array and TRevolution.js

Submitted by Anonymous (not verified) on Wed, 23/11/2011 - 10:51.
i have got a root file which contains array. The root file is able load via javascript but for array i cannot get a histogram .. here is the link may be u can refer. http://www.hecr.tifr.res.in/~noopur/iroot/analysis.html kindly help me. i wanted to post this in forum but my acc is not activated ..

Visualization of arrays

Submitted by axel on Wed, 23/11/2011 - 15:22.

Hi,

Visualization is only implemented for TH1 and TH2, not for TArray. If I were you I'd just store a TH1D or TH1F to the ROOT file instead of a TArrayD or TArrayF.

Cheers, Axel

Hi, Thanks for your reply. In

Submitted by Anonymous (not verified) on Thu, 24/11/2011 - 06:03.
Hi, Thanks for your reply. In fact we are very much interested in using this feature for our experiment for having a web based "data overview" environment and we plan to integrate it with iRoot. Please refer to this: http://root.cern.ch/phpBB3/viewtopic.php?f=12&t=13751 The data is indeed stored in a TH1F (and not a TArray). The name of the histogram is something like Data[0][1]. This is not getting plotted. Should we change the name of the histogram or is there a way out? Thanks once again.

Re: Web-based Data Overview

Submitted by axel on Fri, 25/11/2011 - 16:45.

Hi,

Bertrand has been working on your example :-) Can you try with the newest snapshot from the URL mentioned in the blog post?

Note that we are planning on providing this through a JavaScript API, i.e. you will be able to link in the needed CSS and Javascript files into your (in the future minimal) HTML file. We will post the details once Bertrand publishes this project.

Cheers, Axel.

Excellent!

Submitted by Barth (not verified) on Thu, 17/11/2011 - 12:47.
I am so glad to see this ! Can we expect to get rid of our ALICE online monitoring GUI in the coming year and replace it with a web application ? :) We have a (bit more complex) use case if you wish to hear it... Congratulations again for your work and the very promising demo. Barth

ALICE Online Monitoring?

Submitted by axel on Fri, 18/11/2011 - 23:44.

Hi Barth,

Thanks for your enthusiastic reaction! We definitely need to hear about your use case - can you give a few details be email? Bertrand or I will call you next week to discuss.

Cheers, Axel.

Brilliant

Submitted by Siu (not verified) on Tue, 15/11/2011 - 15:29.
That's a brilliant implementation, very smart use of the range header. On the experimental side Websockets could also be interesting to reduce latency. Congrats

Re: WebSockets

Submitted by axel on Tue, 15/11/2011 - 20:20.

Hi Siu

Thanks for checking out the implementation and thanks even more for your suggestion!

I didn't find WebSockets very tempting for this job: we don't need true bidirectional communication, the amount of chunks requested is rather small, and the standard is still both in flux and not widely implemented. What's your killer argument for WebSockets that I am overlooking?

Cheers, Axel

websockets

Submitted by Siu (not verified) on Wed, 16/11/2011 - 19:09.
I thought the number of needed chunks would increase for bigger files and reducing the latency could benefit the performance, but I don't really know how root stores data internally so maybe it is not the right tool. It would certainly complicate the implementation because it would require server and client support and as you said support in browsers is still far from ideal. Regarding browser support I've been able to use Flash fallbacks to use websockets on "old" browsers. Just rambling around ;) Cheers

Excellent work !!! we wait

Submitted by Jan (not verified) on Mon, 14/11/2011 - 14:52.
Excellent work !!! we wait for further development ... Thanks, Jan

Thanks!

Submitted by axel on Mon, 14/11/2011 - 18:07.

Hi Jan,

Thanks! Bertrand is continuously improving this at the moment, e.g. today. We will announce when it's available for production use - but until then you can always take a current snapshot, no need to wait!

Cheers, Axel.

Hi Axel, long time ! :) +1

Submitted by seb (not verified) on Mon, 14/11/2011 - 14:05.
Hi Axel, long time ! :) +1 (x10e6) thumbs-up ! This is just awesome and definitely would be a very nice step forward in the way we work. So, as I got it ( from your mail and after playing with the code ) only histograms are supported for now. What are the expected improvements ? Is there a link/doc/... one could give a look at on a regular basis to keep inline with the project ? Best, Seb

Documentation

Submitted by axel on Mon, 14/11/2011 - 18:13.

Hi Seb,

Thanks for your thumbs up! Remember that this feature is unreleased - it's too early to expect user-targeted documentation for it :-) Bertrand does comment most of his code, though. Once we release it there will of course be proper documentation.

We did target only histograms for now, expecting that they make up the majority of the actual use cases. All other are "nice to have" but by far not as essential.

Cheers, Axel.

re-hi, thanks four answer. I

Submitted by seb (not verified) on Thu, 17/11/2011 - 11:09.
re-hi, thanks four answer. I would push for getting canvases supported as well as one often wants to compare/look at several distributions at the same time this would definitely be a + cheers, seb

Thanks, Axel! Very nice post

Submitted by Andy (not verified) on Sun, 13/11/2011 - 13:36.
Thanks, Axel! Very nice post and cool stuff! Keep em' coming! Don't forget about TBlackberry please - I love my 9900. :)

Blackberry

Submitted by axel on Mon, 14/11/2011 - 18:23.

Hi Andy!

Thanks for your applause :-) TBlackberry sounds like a high investment into a... ehrm... sinking ship. It can do JavaScript, right? Right!? ;-)

Cheers, Axel.