![]() |
ROOT 6.18/05 Reference Guide |
RDataFrame data source class for reading CSV files.
The RCsvDS class implements a CSV file reader for RDataFrame.
A RDataFrame that reads from a CSV file can be constructed using the factory method ROOT::RDF::MakeCsvDataFrame, which accepts three parameters:
true
). If false
, header names will be automatically generated as Col0, Col1, ..., ColN.The types of the columns in the CSV file are automatically inferred. The supported types are:
true
and false
.These are some formatting rules expected by the RCsvDS implementation:
The current implementation of RCsvDS reads the entire CSV file content into memory before RDataFrame starts processing it. Therefore, before creating a CSV RDataFrame, it is important to check both how much memory is available and the size of the CSV file.
Definition at line 28 of file RCsvDS.hxx.
Public Member Functions | |
RCsvDS (std::string_view fileName, bool readHeaders=true, char delimiter=',', Long64_t linesChunkSize=-1LL) | |
Constructor to create a CSV RDataSource for RDataFrame. More... | |
~RCsvDS () | |
Destructor. More... | |
void | Finalise () |
Convenience method called after concluding an event-loop. More... | |
void | FreeRecords () |
const std::vector< std::string > & | GetColumnNames () const |
Returns a reference to the collection of the dataset's column names. More... | |
std::vector< std::pair< ULong64_t, ULong64_t > > | GetEntryRanges () |
Return ranges of entries to distribute to tasks. More... | |
std::string | GetLabel () |
Return a string representation of the datasource type. More... | |
std::string | GetTypeName (std::string_view colName) const |
Type of a column as a string, e.g. More... | |
bool | HasColumn (std::string_view colName) const |
Checks if the dataset has a certain column. More... | |
bool | SetEntry (unsigned int slot, ULong64_t entry) |
Advance the "cursors" returned by GetColumnReaders to the selected entry for a particular slot. More... | |
void | SetNSlots (unsigned int nSlots) |
Inform RDataSource of the number of processing slots (i.e. More... | |
![]() | |
virtual | ~RDataSource ()=default |
virtual void | Finalise () |
Convenience method called after concluding an event-loop. More... | |
virtual void | FinaliseSlot (unsigned int) |
Convenience method called at the end of the data processing associated to a slot. More... | |
virtual const std::vector< std::string > & | GetColumnNames () const =0 |
Returns a reference to the collection of the dataset's column names. More... | |
template<typename T > | |
std::vector< T ** > | GetColumnReaders (std::string_view columnName) |
Called at most once per column by RDF. More... | |
virtual std::vector< std::pair< ULong64_t, ULong64_t > > | GetEntryRanges ()=0 |
Return ranges of entries to distribute to tasks. More... | |
virtual std::string | GetLabel () |
Return a string representation of the datasource type. More... | |
virtual std::string | GetTypeName (std::string_view) const =0 |
Type of a column as a string, e.g. More... | |
virtual bool | HasColumn (std::string_view) const =0 |
Checks if the dataset has a certain column. More... | |
virtual void | Initialise () |
Convenience method called before starting an event-loop. More... | |
virtual void | InitSlot (unsigned int, ULong64_t) |
Convenience method called at the start of the data processing associated to a slot. More... | |
virtual bool | SetEntry (unsigned int slot, ULong64_t entry)=0 |
Advance the "cursors" returned by GetColumnReaders to the selected entry for a particular slot. More... | |
virtual void | SetNSlots (unsigned int nSlots)=0 |
Inform RDataSource of the number of processing slots (i.e. More... | |
Protected Member Functions | |
std::string | AsString () |
virtual std::string | AsString () |
virtual Record_t | GetColumnReadersImpl (std::string_view name, const std::type_info &)=0 |
type-erased vector of pointers to pointers to column values - one per slot More... | |
Private Types | |
using | ColType_t = char |
Private Member Functions | |
void | FillHeaders (const std::string &) |
void | FillRecord (const std::string &, Record_t &) |
void | GenerateHeaders (size_t) |
std::vector< void * > | GetColumnReadersImpl (std::string_view, const std::type_info &) |
type-erased vector of pointers to pointers to column values - one per slot More... | |
ColType_t | GetType (std::string_view colName) const |
void | InferColTypes (std::vector< std::string > &) |
void | InferType (const std::string &, unsigned int) |
std::vector< std::string > | ParseColumns (const std::string &) |
size_t | ParseValue (const std::string &, std::vector< std::string > &, size_t) |
Private Attributes | |
std::vector< std::deque< bool > > | fBoolEvtValues |
std::vector< std::vector< void * > > | fColAddresses |
std::map< std::string, ColType_t > | fColTypes |
std::list< ColType_t > | fColTypesList |
std::streampos | fDataPos = 0 |
const char | fDelimiter |
std::vector< std::vector< double > > | fDoubleEvtValues |
ULong64_t | fEntryRangesRequested = 0ULL |
std::vector< std::string > | fHeaders |
const Long64_t | fLinesChunkSize |
std::vector< std::vector< Long64_t > > | fLong64EvtValues |
unsigned int | fNSlots = 0U |
ULong64_t | fProcessedLines = 0ULL |
bool | fReadHeaders = false |
std::vector< Record_t > | fRecords |
std::ifstream | fStream |
std::vector< std::vector< std::string > > | fStringEvtValues |
Static Private Attributes | |
static TRegexp | doubleRegex1 |
static TRegexp | doubleRegex2 |
static TRegexp | falseRegex |
static const std::map< ColType_t, std::string > | fgColTypeMap |
static TRegexp | intRegex |
static TRegexp | trueRegex |
Additional Inherited Members | |
![]() | |
using | Record_t = std::vector< void * > |
#include <ROOT/RCsvDS.hxx>
|
private |
Definition at line 32 of file RCsvDS.hxx.
ROOT::RDF::RCsvDS::RCsvDS | ( | std::string_view | fileName, |
bool | readHeaders = true , |
||
char | delimiter = ',' , |
||
Long64_t | linesChunkSize = -1LL |
||
) |
Constructor to create a CSV RDataSource for RDataFrame.
[in] | fileName | Path of the CSV file. |
[in] | readHeaders | true if the CSV file contains headers as first row, false otherwise (default true ). |
[in] | delimiter | Delimiter character (default ','). |
Definition at line 257 of file RCsvDS.cxx.
ROOT::RDF::RCsvDS::~RCsvDS | ( | ) |
Destructor.
Definition at line 324 of file RCsvDS.cxx.
|
protectedvirtual |
Reimplemented from ROOT::RDF::RDataSource.
Definition at line 89 of file RCsvDS.cxx.
|
private |
Definition at line 104 of file RCsvDS.cxx.
Definition at line 112 of file RCsvDS.cxx.
|
virtual |
Convenience method called after concluding an event-loop.
See Initialise for more details.
Reimplemented from ROOT::RDF::RDataSource.
Definition at line 329 of file RCsvDS.cxx.
void ROOT::RDF::RCsvDS::FreeRecords | ( | ) |
Definition at line 293 of file RCsvDS.cxx.
|
private |
Definition at line 147 of file RCsvDS.cxx.
|
virtual |
Returns a reference to the collection of the dataset's column names.
Implements ROOT::RDF::RDataSource.
Definition at line 338 of file RCsvDS.cxx.
|
privatevirtual |
type-erased vector of pointers to pointers to column values - one per slot
Implements ROOT::RDF::RDataSource.
Definition at line 154 of file RCsvDS.cxx.
Return ranges of entries to distribute to tasks.
They are required to be contiguous intervals with no entries skipped. Supposing a dataset with nEntries, the intervals must start at 0 and end at nEntries, e.g. [0-5],[5-10] for 10 entries. This function will be invoked repeatedly by RDataFrame as it needs additional entries to process. The same entry range should not be returned more than once. Returning an empty collection of ranges signals to RDataFrame that the processing can stop.
Implements ROOT::RDF::RDataSource.
Definition at line 343 of file RCsvDS.cxx.
|
virtual |
Return a string representation of the datasource type.
The returned string will be used by ROOT::RDF::SaveGraph() to represent the datasource in the visualization of the computation graph. Concrete datasources can override the default implementation.
Reimplemented from ROOT::RDF::RDataSource.
Definition at line 457 of file RCsvDS.cxx.
|
private |
Definition at line 388 of file RCsvDS.cxx.
|
virtual |
Type of a column as a string, e.g.
GetTypeName("x") == "double"
. Required for jitting e.g. df.Filter("x>0")
.
[in] | columnName | The name of the column |
Implements ROOT::RDF::RDataSource.
Definition at line 399 of file RCsvDS.cxx.
|
virtual |
Checks if the dataset has a certain column.
[in] | columnName | The name of the column |
Implements ROOT::RDF::RDataSource.
Definition at line 404 of file RCsvDS.cxx.
|
private |
Definition at line 186 of file RCsvDS.cxx.
|
private |
Definition at line 195 of file RCsvDS.cxx.
|
private |
Definition at line 215 of file RCsvDS.cxx.
|
private |
Definition at line 226 of file RCsvDS.cxx.
|
virtual |
Advance the "cursors" returned by GetColumnReaders to the selected entry for a particular slot.
[in] | slot | The data processing slot that needs to be considered |
[in] | entry | The entry which needs to be pointed to by the reader pointers Slots are adopted to accommodate parallel data processing. Different workers will loop over different ranges and will be labelled by different "slot" values. Returns true if the entry has to be processed, false otherwise. |
Implements ROOT::RDF::RDataSource.
Definition at line 409 of file RCsvDS.cxx.
|
virtual |
Inform RDataSource of the number of processing slots (i.e.
worker threads) used by the associated RDataFrame. Slots numbers are used to simplify parallel execution: RDataFrame guarantees that different threads will always pass different slot values when calling methods concurrently.
Implements ROOT::RDF::RDataSource.
Definition at line 440 of file RCsvDS.cxx.
|
staticprivate |
Definition at line 55 of file RCsvDS.hxx.
|
staticprivate |
Definition at line 55 of file RCsvDS.hxx.
|
staticprivate |
Definition at line 55 of file RCsvDS.hxx.
|
private |
Definition at line 53 of file RCsvDS.hxx.
|
private |
Definition at line 46 of file RCsvDS.hxx.
|
private |
Definition at line 44 of file RCsvDS.hxx.
|
private |
Definition at line 45 of file RCsvDS.hxx.
|
private |
Definition at line 35 of file RCsvDS.hxx.
|
private |
Definition at line 39 of file RCsvDS.hxx.
|
private |
Definition at line 48 of file RCsvDS.hxx.
|
private |
Definition at line 41 of file RCsvDS.hxx.
|
staticprivate |
Definition at line 33 of file RCsvDS.hxx.
|
private |
Definition at line 43 of file RCsvDS.hxx.
|
private |
Definition at line 40 of file RCsvDS.hxx.
|
private |
Definition at line 49 of file RCsvDS.hxx.
|
private |
Definition at line 37 of file RCsvDS.hxx.
|
private |
Definition at line 42 of file RCsvDS.hxx.
|
private |
Definition at line 36 of file RCsvDS.hxx.
|
private |
Definition at line 47 of file RCsvDS.hxx.
|
private |
Definition at line 38 of file RCsvDS.hxx.
|
private |
Definition at line 50 of file RCsvDS.hxx.
|
staticprivate |
Definition at line 55 of file RCsvDS.hxx.
|
staticprivate |
Definition at line 55 of file RCsvDS.hxx.