2013
Nouanesengsy, Boonthanome; Patchett, John; Ahrens, James; Bauer, Andrew; Chaudhary, Aashish; Geveci, Berk; Miller, Ross; Shipman, Galen; Williams, Dean N
Optimizing File Access Patterns through the Spatio-Temporal Pipeline for Parallel Visualization and Analysis Technical Report
2013, (LA-UR-pending).
Abstract | Links | BibTeX | Tags: Data Analysis, I/O, Modeling, Parallel Analysis, Parallel Techniques, Parallel Visualization, Spatio-Temporal Pipeline, visualization
@techreport{Nouanesengsy2013,
title = {Optimizing File Access Patterns through the Spatio-Temporal Pipeline for Parallel Visualization and Analysis},
author = {Boonthanome Nouanesengsy and John Patchett and James Ahrens and Andrew Bauer and Aashish Chaudhary and Berk Geveci and Ross Miller and Galen Shipman and Dean N Williams},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/06/OptimizingFileAccessPatternsThroughTheSpatio-TemporalPipelineForParallelVisualizationAndAnalysis.pdf},
year = {2013},
date = {2013-10-13},
abstract = {As computational resources have become more powerful over time, availability of large-scale data has exploded, with datasets greatly increasing their spatial and temporal resolutions. For many years now, I/O read time has been recognized as the primary bottleneck for parallel visualization and analysis of large-scale data. Read times ultimately depends on how the file is stored and the file access pattern used to read the file. In this paper, we introduce a model which can estimate the read time for a file stored in a parallel filesystem when given the file access pattern. The type of parallel decomposition used directly dictates what the file access pattern will be. The spatio-temporal pipeline is used to give greater flexibility to the file access pattern used. The spatio-temporal pipeline combines both spatial and temporal parallelism to create a parallel decomposition for a task. Within the spatio-temporal pipeline, all available processes are divided into groups called time compartments. Temporal parallelism is utilized as different timesteps are independently processed by separate time compartments, and spatial parallelism is used to divide each timestep over all processes within a time compartment. The ratio between spatial and temporal parallelism is controlled by adjusting the size of a time compartment. Using the model, we were able to configure the spatio-temporal pipeline to create optimized read access patterns, resulting in a speedup factor of approximately 400 over traditional file access patterns.},
note = {LA-UR-pending},
keywords = {Data Analysis, I/O, Modeling, Parallel Analysis, Parallel Techniques, Parallel Visualization, Spatio-Temporal Pipeline, visualization},
pubstate = {published},
tppubtype = {techreport}
}
As computational resources have become more powerful over time, availability of large-scale data has exploded, with datasets greatly increasing their spatial and temporal resolutions. For many years now, I/O read time has been recognized as the primary bottleneck for parallel visualization and analysis of large-scale data. Read times ultimately depends on how the file is stored and the file access pattern used to read the file. In this paper, we introduce a model which can estimate the read time for a file stored in a parallel filesystem when given the file access pattern. The type of parallel decomposition used directly dictates what the file access pattern will be. The spatio-temporal pipeline is used to give greater flexibility to the file access pattern used. The spatio-temporal pipeline combines both spatial and temporal parallelism to create a parallel decomposition for a task. Within the spatio-temporal pipeline, all available processes are divided into groups called time compartments. Temporal parallelism is utilized as different timesteps are independently processed by separate time compartments, and spatial parallelism is used to divide each timestep over all processes within a time compartment. The ratio between spatial and temporal parallelism is controlled by adjusting the size of a time compartment. Using the model, we were able to configure the spatio-temporal pipeline to create optimized read access patterns, resulting in a speedup factor of approximately 400 over traditional file access patterns.
2009
Patchett, John; Ahrens, James; Ahern, Sean; Pugmire, David
Parallel visualization and analysis with ParaView on a Cray Xt4 Journal Article
In: Cray User Group, 2009, (LA-UR-10-02238).
Abstract | Links | BibTeX | Tags: Parallel Visualization, ParaView
@article{patchett2009parallel,
title = {Parallel visualization and analysis with ParaView on a Cray Xt4},
author = {John Patchett and James Ahrens and Sean Ahern and David Pugmire},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/06/ParallelVisualizationAndAnalysisWithParaViewOnACrayXT4.pdf},
year = {2009},
date = {2009-01-01},
journal = {Cray User Group},
abstract = {Scientific data sets produced by modern supercomputers like ORNL’s Cray XT 4, Jaguar, can be extremely large, making visualization and analysis more difficult as moving large resultant data to dedicated analysis systems can be pro- hibitively expensive. We share our continuing work of integrating a parallel visu- alization system, ParaView, on ORNL’s Jaguar system and our efforts to enable extreme scale interactive data visualization and analysis. We will discuss porting challenges and present performance numbers.},
note = {LA-UR-10-02238},
keywords = {Parallel Visualization, ParaView},
pubstate = {published},
tppubtype = {article}
}
Scientific data sets produced by modern supercomputers like ORNL’s Cray XT 4, Jaguar, can be extremely large, making visualization and analysis more difficult as moving large resultant data to dedicated analysis systems can be pro- hibitively expensive. We share our continuing work of integrating a parallel visu- alization system, ParaView, on ORNL’s Jaguar system and our efforts to enable extreme scale interactive data visualization and analysis. We will discuss porting challenges and present performance numbers.
: . .
1.
Nouanesengsy, Boonthanome; Patchett, John; Ahrens, James; Bauer, Andrew; Chaudhary, Aashish; Geveci, Berk; Miller, Ross; Shipman, Galen; Williams, Dean N
Optimizing File Access Patterns through the Spatio-Temporal Pipeline for Parallel Visualization and Analysis Technical Report
2013, (LA-UR-pending).
@techreport{Nouanesengsy2013,
title = {Optimizing File Access Patterns through the Spatio-Temporal Pipeline for Parallel Visualization and Analysis},
author = {Boonthanome Nouanesengsy and John Patchett and James Ahrens and Andrew Bauer and Aashish Chaudhary and Berk Geveci and Ross Miller and Galen Shipman and Dean N Williams},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/06/OptimizingFileAccessPatternsThroughTheSpatio-TemporalPipelineForParallelVisualizationAndAnalysis.pdf},
year = {2013},
date = {2013-10-13},
abstract = {As computational resources have become more powerful over time, availability of large-scale data has exploded, with datasets greatly increasing their spatial and temporal resolutions. For many years now, I/O read time has been recognized as the primary bottleneck for parallel visualization and analysis of large-scale data. Read times ultimately depends on how the file is stored and the file access pattern used to read the file. In this paper, we introduce a model which can estimate the read time for a file stored in a parallel filesystem when given the file access pattern. The type of parallel decomposition used directly dictates what the file access pattern will be. The spatio-temporal pipeline is used to give greater flexibility to the file access pattern used. The spatio-temporal pipeline combines both spatial and temporal parallelism to create a parallel decomposition for a task. Within the spatio-temporal pipeline, all available processes are divided into groups called time compartments. Temporal parallelism is utilized as different timesteps are independently processed by separate time compartments, and spatial parallelism is used to divide each timestep over all processes within a time compartment. The ratio between spatial and temporal parallelism is controlled by adjusting the size of a time compartment. Using the model, we were able to configure the spatio-temporal pipeline to create optimized read access patterns, resulting in a speedup factor of approximately 400 over traditional file access patterns.},
note = {LA-UR-pending},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
As computational resources have become more powerful over time, availability of large-scale data has exploded, with datasets greatly increasing their spatial and temporal resolutions. For many years now, I/O read time has been recognized as the primary bottleneck for parallel visualization and analysis of large-scale data. Read times ultimately depends on how the file is stored and the file access pattern used to read the file. In this paper, we introduce a model which can estimate the read time for a file stored in a parallel filesystem when given the file access pattern. The type of parallel decomposition used directly dictates what the file access pattern will be. The spatio-temporal pipeline is used to give greater flexibility to the file access pattern used. The spatio-temporal pipeline combines both spatial and temporal parallelism to create a parallel decomposition for a task. Within the spatio-temporal pipeline, all available processes are divided into groups called time compartments. Temporal parallelism is utilized as different timesteps are independently processed by separate time compartments, and spatial parallelism is used to divide each timestep over all processes within a time compartment. The ratio between spatial and temporal parallelism is controlled by adjusting the size of a time compartment. Using the model, we were able to configure the spatio-temporal pipeline to create optimized read access patterns, resulting in a speedup factor of approximately 400 over traditional file access patterns.
2.
Patchett, John; Ahrens, James; Ahern, Sean; Pugmire, David
Parallel visualization and analysis with ParaView on a Cray Xt4 Journal Article
In: Cray User Group, 2009, (LA-UR-10-02238).
@article{patchett2009parallel,
title = {Parallel visualization and analysis with ParaView on a Cray Xt4},
author = {John Patchett and James Ahrens and Sean Ahern and David Pugmire},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/06/ParallelVisualizationAndAnalysisWithParaViewOnACrayXT4.pdf},
year = {2009},
date = {2009-01-01},
journal = {Cray User Group},
abstract = {Scientific data sets produced by modern supercomputers like ORNL’s Cray XT 4, Jaguar, can be extremely large, making visualization and analysis more difficult as moving large resultant data to dedicated analysis systems can be pro- hibitively expensive. We share our continuing work of integrating a parallel visu- alization system, ParaView, on ORNL’s Jaguar system and our efforts to enable extreme scale interactive data visualization and analysis. We will discuss porting challenges and present performance numbers.},
note = {LA-UR-10-02238},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Scientific data sets produced by modern supercomputers like ORNL’s Cray XT 4, Jaguar, can be extremely large, making visualization and analysis more difficult as moving large resultant data to dedicated analysis systems can be pro- hibitively expensive. We share our continuing work of integrating a parallel visu- alization system, ParaView, on ORNL’s Jaguar system and our efforts to enable extreme scale interactive data visualization and analysis. We will discuss porting challenges and present performance numbers.