Quoting HP Wei <hp@rentec.com>: > Our_database_package is for the custom software that is currently > used in our company. The database is compressed extensively. > It also stores some indexes info for fast accessing. > Even with the index info stored in the file, the file size > is still about 30%-50% smaller than the root_file counterpart > (one tree for one stock) for a given month. Not surprising to me. > Note: this exercise is not to show off (or advertizing) our > custom_database package. We are investigating whether we > can utilize or integrate ROOT. If you are deciding b/w ROOT and your own database format, I would recommend choose your database format. If you are trying to sell something this is the wrong forum anyway .... > The existing system is thus the obvious benchmark target. > I am wondering if there are some 'tricks' (hidden functionalities) > in ROOT to improve the data storage and accessing performance. maybe decompressing=OFF but file sizes will explode. > Valeri mentioned about TTable class. > Maybe that is another thing to try. > Anyone knows the fundamental difference between TTable and TTree > in terms of how they organize the data on disk ?? Tables are a little more efficient that Trees but not much, just smaller overhead for each table. That's because describing a table takes less space than describing a tree. But the elements don't get smaller. Now if you try saving tables of, say, integers, rather than doubles it will get more efficient, but nowhere as efficient as a database of integers. Why? write some small sample programs you will see that each additional integer field does not add 2 more bytes per record on disk, it's much more than that... You could also try to save the data in binary rather than as a ROOT object but it will get complicated and difficult to mantain, it's much easier to program a ROOT<->database interface. But anyway, if anybody else knows a way to save large tick data files in ROOT at least as efficiently as a proprietary database format, then I am interested in hearing from you. I am always looking to improve performance.
This archive was generated by hypermail 2b29 : Sat Jan 04 2003 - 23:51:20 MET