Hi Eddy, Thanks for this posting. I should have mentionned your option of having Trees in parallel, of course the best solution. Rene Brun Eddy Offermann wrote: > > Hi Anton, > > Over the weekend I saw that you used an example of data mining in the financial > environment to argue your point of view concerning insertion/deletion > of tree entries. > Well let's persue that a little bit more. So data is coming from providers > like Reuters, Bloomberg. CantorFitz, GL ...... The list can be extended quite > a bit. Now the first thing to wonder about is why so many. Well many > will provide tick data on the same commodity/equity/option but unfortunately > not agree/ have data missing. > > That is the real-time aspect of it. In addition you want to store historical > data which will come from again other vendors. > > If you want to build the "perfect" database for commodity x, you will most > likely use statistical tests to decide which feed is right and store its > entry in your "perfect" database (who is right is time dependent). > > To make a long story short: > 1) You will have to store all the incoming information as is, you do not > want to alter the "raw" data > 2) Your data used for simulation / prediction is a combination of the > different sources. You might change the selection algorithm over time. > 3) Records from vendors changing previous data constitute only a small > part of your data. So why not have besides your usual tree another > tree containing "corrections". Each time you load data from the tree > you check if corrections are available. If the number gets too large > you will create a new tree incorporating them. > > I have seen you raise again the point of manpower on the ROOT project but in the > same breath you mention inefficient management of these large-scale experiments. > So what do you want ? > > It is only natural that the ROOT team listens closely to the wishes of the > physics community. An easy way to put your mark on the project is to open > a can with programmers at Fortis-bank and implement your vision. Because > let's face it at the moment you and I have a free ride using the ROOT team. > > Eddy Offermann > Renaissance Technologies Corp. > Route 25A, East Setauket NY 11733 > e-mail: eddy@rentec.com > http://www.rentec.com
This archive was generated by hypermail 2b29 : Tue Jan 01 2002 - 17:50:38 MET