2015
Adhinarayanan, Vignesh
Performance, Power and Energy of In-situ and Post-processing Visualization: A Case Study in Climate Simulation Presentation
05.10.2015, (LA-UR-15-27749).
Abstract | Links | BibTeX | Tags: energy, in-situ, performance, post-processing, power
@misc{Adhinarayanan2015,
title = {Performance, Power and Energy of In-situ and Post-processing Visualization: A Case Study in Climate Simulation},
author = {Vignesh Adhinarayanan},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/07/PerformancePowerAndEnergyOfInSituAndPostProcessingVisualization.pdf},
year = {2015},
date = {2015-10-05},
abstract = {This presentation summarizes a summer study of the performance, power, and energy trade-offs among traditional post-processing, modern post-processing, and in-situ visualization pipelines. It includes both detailed sub-component level power measurements within a node to gain detailed insights and measurements at scale to understand problems unique to big supercomputers.},
note = {LA-UR-15-27749},
keywords = {energy, in-situ, performance, post-processing, power},
pubstate = {published},
tppubtype = {presentation}
}
This presentation summarizes a summer study of the performance, power, and energy trade-offs among traditional post-processing, modern post-processing, and in-situ visualization pipelines. It includes both detailed sub-component level power measurements within a node to gain detailed insights and measurements at scale to understand problems unique to big supercomputers.
2013
Su, Yu; Agrawal, Gagan; Woodring, Jonathan; Myers, Kary; Wendelberger, Joanne; Ahrens, James
Taming massive distributed datasets: data sampling using bitmap indices Proceedings Article
In: Proceedings of the 22nd international symposium on High-performance parallel and distributed computing, pp. 13–24, ACM 2013, (LA-UR-pending).
Abstract | Links | BibTeX | Tags: bitmap indices, database applications, database management, performance
@inproceedings{su2013taming,
title = {Taming massive distributed datasets: data sampling using bitmap indices},
author = {Yu Su and Gagan Agrawal and Jonathan Woodring and Kary Myers and Joanne Wendelberger and James Ahrens},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/06/TamingMassiveDistributedDatasetsDataSamplingUsingBitmapIndices.pdf},
year = {2013},
date = {2013-01-01},
booktitle = {Proceedings of the 22nd international symposium on High-performance parallel and distributed computing},
pages = {13--24},
organization = {ACM},
abstract = {In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting. We observe that to allow subsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: 1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a value-based and/or a dimension-based subsetting predicate, and 2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.},
note = {LA-UR-pending},
keywords = {bitmap indices, database applications, database management, performance},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting. We observe that to allow subsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: 1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a value-based and/or a dimension-based subsetting predicate, and 2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.
: . .
1.
Adhinarayanan, Vignesh
Performance, Power and Energy of In-situ and Post-processing Visualization: A Case Study in Climate Simulation Presentation
05.10.2015, (LA-UR-15-27749).
@misc{Adhinarayanan2015,
title = {Performance, Power and Energy of In-situ and Post-processing Visualization: A Case Study in Climate Simulation},
author = {Vignesh Adhinarayanan},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/07/PerformancePowerAndEnergyOfInSituAndPostProcessingVisualization.pdf},
year = {2015},
date = {2015-10-05},
abstract = {This presentation summarizes a summer study of the performance, power, and energy trade-offs among traditional post-processing, modern post-processing, and in-situ visualization pipelines. It includes both detailed sub-component level power measurements within a node to gain detailed insights and measurements at scale to understand problems unique to big supercomputers.},
note = {LA-UR-15-27749},
keywords = {},
pubstate = {published},
tppubtype = {presentation}
}
This presentation summarizes a summer study of the performance, power, and energy trade-offs among traditional post-processing, modern post-processing, and in-situ visualization pipelines. It includes both detailed sub-component level power measurements within a node to gain detailed insights and measurements at scale to understand problems unique to big supercomputers.
2.
Su, Yu; Agrawal, Gagan; Woodring, Jonathan; Myers, Kary; Wendelberger, Joanne; Ahrens, James
Taming massive distributed datasets: data sampling using bitmap indices Proceedings Article
In: Proceedings of the 22nd international symposium on High-performance parallel and distributed computing, pp. 13–24, ACM 2013, (LA-UR-pending).
@inproceedings{su2013taming,
title = {Taming massive distributed datasets: data sampling using bitmap indices},
author = {Yu Su and Gagan Agrawal and Jonathan Woodring and Kary Myers and Joanne Wendelberger and James Ahrens},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/06/TamingMassiveDistributedDatasetsDataSamplingUsingBitmapIndices.pdf},
year = {2013},
date = {2013-01-01},
booktitle = {Proceedings of the 22nd international symposium on High-performance parallel and distributed computing},
pages = {13--24},
organization = {ACM},
abstract = {In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting. We observe that to allow subsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: 1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a value-based and/or a dimension-based subsetting predicate, and 2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.},
note = {LA-UR-pending},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting. We observe that to allow subsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: 1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a value-based and/or a dimension-based subsetting predicate, and 2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.