Contents

Class gdf_data_provider

Inheritance Relationships

Base Type

Class Documentation

class ral::io::gdf_data_provider : public ral::io::data_provider

can generate a series of data_handles from gdfs that are provided when it goes out of scope it will close release any gdfs

Public Functions

gdf_data_provider(std::vector<ral::frame::BlazingTableView> table_views, std::vector<std::map<std::string, std::string>> column_values)
virtual std::shared_ptr<data_provider> clone() override
virtual ~gdf_data_provider()
virtual bool has_next()

tells us if there are more files in the list of uris to be provided

virtual void reset()

Resets current_file to 0

virtual data_handle get_next(bool open_file = true)

gets a randomaccessfile to the uri at file_uris[current_file] and advances current_file by 1 open_file = true will actually open the file and return a std::shared_ptr<arrow::io::RandomAccessFile>. If its false it will return a nullptr

std::vector<std::string> get_errors()

returns an empty vector. Used for compatiblity with apis that open files.

virtual std::vector<data_handle> get_some(std::size_t num_files, bool open_file = true)

Tries to get up to num_files data_handles. We use this instead of a get_all() because if there are too many files, trying to get too many file handles will cause a crash. Using get_some() forces breaking up the process of getting file_handles. open_file = true will actually open the file and return a std::shared_ptr<arrow::io::RandomAccessFile>. If its false it will return a nullptr

virtual void close_file_handles()

Does nothing. Used for compatiblity with apis that open files.

Closes currently open set of file handles maintained by the provider

virtual size_t get_num_handles()

Get the number of data_handles that will be provided.

Class data_provider Class gdf_parser