User-Defined Tensor Data Analysis

1. Introduction.- 1.1 Lessons from Big Data Systems.- 1.2 Data Model.- 1. 3 Programming Model High-Performance Data Analysis for Science.- 2. FasTensor Programming Model.- 2.1 Introduction to Tensor Data Model.- 2.2 FasTensor Programming Model.- 2.2.1 Stencils.- 2.2.2 Chunks.- 2.2.3 Overlap.- 2.2.4 Operator: Transform.- 2.2.5 FasTensor Execution Engine.- 2.2.6 FasTensor Scientific Computing Use Cases.- 2.3 Summary.- Illustrated FasTensor User Interface.- 3.1 An Example.- 3.2 The Stencil Class.- 3.2.1 Constructors of the Stencil.- 3.2.2 Parenthesis operator () and ReadPoint.- 3.2.3 SetShape and GetShape.- 3.2.4 SetValue and GetValue.- 3.2.5 ReadNeighbors and WriteNeighbors.- 3.2.6 GetOffsetUpper and GetOffsetLower.- 3.2.7 GetChunkID.- 3.2.8 GetGlobalIndex and GetLocalIndex.- 3.2.9 Exercise of the Stencil class.- 3.3 The Array Class.- 3.3.1 Constructors of Array.- 3.3.2 SetChunkSize, SetChunkSizeByMem, SetChunkSizeByDim, and GetChunkSize.- 3.3.3 SetOverlapSize, SetOverlapSizeByDetection,GetOverlapSize, SetOverlapPadding, and SyncOverlap.- 3.3.4               Transform.- 3.3.5 SetStride and GetStride.- 3.3.6 AppendAttribute, InsertAttribute, GetAttribute and EraseAttribute.- 3.3.7 SetEndpoint and GetEndpoint.-  3.3.8 ControlEndpoint.- 3.3.9.- ReadArray and WriteArray.- 3.3.10 SetTag and GetTag.- 3.3.11 GetArraySize and SetArraySize.- 3.3.12 Backup and Restore.- 3.3.13 CreateVisFile.- 3.3.14 ReportCost.- 3.3.15 EP_DIR Endpoint.- 3.3.16 EP_HDF5 and Other Endpoints.- Other Functions in FasTensor.- 3.4.1 FT_Init.- 3.4.2 FT_Finalize.- 3.4.3 Data types in FasTensor.- 4. FasTensor in Real Scientific Applications.- 4.1 DAS: Distributed Acoustic Sensing.- 4.2 VPIC: Vector Particle-In-Cell.- Appendix.- A.1 Installation Guide of FasTensor.- A.2 How to Develop a New Endpoint Protocol.- Alphabetical Index.- Bibliography.- References.



 

Dr. Bin Dong is a Research Scientist in Lawrence Berkeley National Laboratory in Berkeley, California, USA. Bin has the Ph.D degree in computing science and technology. Bin has wide research interests in big scientific data analysis, parallel computing, parallel I/O, machine learning, etc. He has co-authored more than 62 technical publications.
Dr. Kesheng Wu is a Senior Scientist at Lawrence Berkeley National Laboratory. He works extensively on data management, data analysis, and scientific computing. He is the developer of a number of widely used algorithms including FastBit bitmap indexes for querying large scientific datasets, Thick-Restart Lanczos (TRLan) algorithm for solving eigenvalue problems, and IDEALEM for statistical data reduction and feature extraction. He has co-authored more than 200 technical publications.

Dr. Suren Byna is a Computer Scientist in the Scientific Data Management (SDM) Group at Lawrence Berkeley National Laboratory in Berkeley, California, USA. His research interests are in scalable scientific data management. More specifically, he works on optimizing parallel I/O and on developing systems for managing scientific data. He leads the ExaIO project in the Exascale Computing Project (ECP) that contributes advanced I/O features to HDF5 and develops a new file system called UnifyFS. He also leads efforts that develop object-centric data management systems (Proactive Data Containers - PDC) and experimental and observational data (EOD) management strategies. He has co-authored more than 150 technical publications.