2015
Pulido, Jesus
Survey and Analysis of Multiresolution Methods for Turbulence Data Presentation
05.10.2015, (LA-UR-15-27727).
Abstract | Links | BibTeX | Tags: Multi-resolution, turbulence
@misc{Pulido2015,
title = {Survey and Analysis of Multiresolution Methods for Turbulence Data},
author = {Jesus Pulido},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/08/Survey_and_Analysis_of_Multiresolution_Methods_for_Turbulence_Data.pdf},
year = {2015},
date = {2015-10-05},
abstract = {The first section of this presentation summarizes summer work that analyzed and compared the effectiveness of Haar, Biorthogonal B-spline, and Daubechies wavelets, Coiflets, Surfacelets and Curvelets with respect to direct numerical simulation (DNS) data, visualized and evaluated numerical accuracy of original data and derived quantities using limited amounts of coefficients on reconstruction, and identified strengths and weaknesses of each technique and gave recommendations on the usage of these multi-resolution methods.},
note = {LA-UR-15-27727},
keywords = {Multi-resolution, turbulence},
pubstate = {published},
tppubtype = {presentation}
}
The first section of this presentation summarizes summer work that analyzed and compared the effectiveness of Haar, Biorthogonal B-spline, and Daubechies wavelets, Coiflets, Surfacelets and Curvelets with respect to direct numerical simulation (DNS) data, visualized and evaluated numerical accuracy of original data and derived quantities using limited amounts of coefficients on reconstruction, and identified strengths and weaknesses of each technique and gave recommendations on the usage of these multi-resolution methods.
2014
Su, Yu; Agrawal, Gagan; Woodring, Jonathan; Myers, Kary; Wendelberger, Joanne; Ahrens, James
Effective and efficient data sampling using bitmap indices Journal Article
In: Cluster Computing, pp. 1-20, 2014, ISSN: 1386-7857, (LA-UR-pending).
Abstract | Links | BibTeX | Tags: big data, Bitmap indexing, data sampling, Multi-resolution, parallel processing
@article{,
title = {Effective and efficient data sampling using bitmap indices},
author = {Yu Su and Gagan Agrawal and Jonathan Woodring and Kary Myers and Joanne Wendelberger and James Ahrens},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/06/EffectiveAndEfficientDataSamplingUsingBitmapIndeces.pdf},
doi = {10.1007/s10586-014-0360-5},
issn = {1386-7857},
year = {2014},
date = {2014-01-01},
journal = {Cluster Computing},
pages = {1-20},
publisher = {Springer US},
abstract = {With growing computational capabilities of parallel machines, scientific simulations are being performed at finer spatial and temporal scales, leading to a data explosion. The growing sizes are making it extremely hard to store, manage, disseminate, analyze, and visualize these datasets, especially as neither the memory capacity of parallel machines, memory access speeds, nor disk bandwidths are increasing at the same rate as the computing power. Sampling can be an effective technique to address the above challenges, but it is extremely important to ensure that dataset characteristics are preserved, and the loss of accuracy is within acceptable levels. In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting.We observe that to allowsubsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: (1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a valuebased and/or a dimension-based subsetting predicate, and (2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.},
note = {LA-UR-pending},
keywords = {big data, Bitmap indexing, data sampling, Multi-resolution, parallel processing},
pubstate = {published},
tppubtype = {article}
}
With growing computational capabilities of parallel machines, scientific simulations are being performed at finer spatial and temporal scales, leading to a data explosion. The growing sizes are making it extremely hard to store, manage, disseminate, analyze, and visualize these datasets, especially as neither the memory capacity of parallel machines, memory access speeds, nor disk bandwidths are increasing at the same rate as the computing power. Sampling can be an effective technique to address the above challenges, but it is extremely important to ensure that dataset characteristics are preserved, and the loss of accuracy is within acceptable levels. In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting.We observe that to allowsubsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: (1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a valuebased and/or a dimension-based subsetting predicate, and (2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.
: . .
1.
Pulido, Jesus
Survey and Analysis of Multiresolution Methods for Turbulence Data Presentation
05.10.2015, (LA-UR-15-27727).
@misc{Pulido2015,
title = {Survey and Analysis of Multiresolution Methods for Turbulence Data},
author = {Jesus Pulido},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/08/Survey_and_Analysis_of_Multiresolution_Methods_for_Turbulence_Data.pdf},
year = {2015},
date = {2015-10-05},
abstract = {The first section of this presentation summarizes summer work that analyzed and compared the effectiveness of Haar, Biorthogonal B-spline, and Daubechies wavelets, Coiflets, Surfacelets and Curvelets with respect to direct numerical simulation (DNS) data, visualized and evaluated numerical accuracy of original data and derived quantities using limited amounts of coefficients on reconstruction, and identified strengths and weaknesses of each technique and gave recommendations on the usage of these multi-resolution methods.},
note = {LA-UR-15-27727},
keywords = {},
pubstate = {published},
tppubtype = {presentation}
}
The first section of this presentation summarizes summer work that analyzed and compared the effectiveness of Haar, Biorthogonal B-spline, and Daubechies wavelets, Coiflets, Surfacelets and Curvelets with respect to direct numerical simulation (DNS) data, visualized and evaluated numerical accuracy of original data and derived quantities using limited amounts of coefficients on reconstruction, and identified strengths and weaknesses of each technique and gave recommendations on the usage of these multi-resolution methods.
2.
Su, Yu; Agrawal, Gagan; Woodring, Jonathan; Myers, Kary; Wendelberger, Joanne; Ahrens, James
Effective and efficient data sampling using bitmap indices Journal Article
In: Cluster Computing, pp. 1-20, 2014, ISSN: 1386-7857, (LA-UR-pending).
@article{,
title = {Effective and efficient data sampling using bitmap indices},
author = {Yu Su and Gagan Agrawal and Jonathan Woodring and Kary Myers and Joanne Wendelberger and James Ahrens},
url = {http://datascience.dsscale.org/wp-content/uploads/2016/06/EffectiveAndEfficientDataSamplingUsingBitmapIndeces.pdf},
doi = {10.1007/s10586-014-0360-5},
issn = {1386-7857},
year = {2014},
date = {2014-01-01},
journal = {Cluster Computing},
pages = {1-20},
publisher = {Springer US},
abstract = {With growing computational capabilities of parallel machines, scientific simulations are being performed at finer spatial and temporal scales, leading to a data explosion. The growing sizes are making it extremely hard to store, manage, disseminate, analyze, and visualize these datasets, especially as neither the memory capacity of parallel machines, memory access speeds, nor disk bandwidths are increasing at the same rate as the computing power. Sampling can be an effective technique to address the above challenges, but it is extremely important to ensure that dataset characteristics are preserved, and the loss of accuracy is within acceptable levels. In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting.We observe that to allowsubsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: (1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a valuebased and/or a dimension-based subsetting predicate, and (2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.},
note = {LA-UR-pending},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
With growing computational capabilities of parallel machines, scientific simulations are being performed at finer spatial and temporal scales, leading to a data explosion. The growing sizes are making it extremely hard to store, manage, disseminate, analyze, and visualize these datasets, especially as neither the memory capacity of parallel machines, memory access speeds, nor disk bandwidths are increasing at the same rate as the computing power. Sampling can be an effective technique to address the above challenges, but it is extremely important to ensure that dataset characteristics are preserved, and the loss of accuracy is within acceptable levels. In this paper, we address the data explosion problems by developing a novel sampling approach, and implementing it in a flexible system that supports server-side sampling and data subsetting.We observe that to allowsubsetting over scientific datasets, data repositories are likely to use an indexing technique. Among these techniques, we see that bitmap indexing can not only effectively support subsetting over scientific datasets, but can also help create samples that preserve both value and spatial distributions over scientific datasets. We have developed algorithms for using bitmap indices to sample datasets. We have also shown how only a small amount of additional metadata stored with bitvectors can help assess loss of accuracy with a particular subsampling level. Some of the other properties of this novel approach include: (1) sampling can be flexibly applied to a subset of the original dataset, which may be specified using a valuebased and/or a dimension-based subsetting predicate, and (2) no data reorganization is needed, once bitmap indices have been generated. We have extensively evaluated our method with different types of datasets and applications, and demonstrated the effectiveness of our approach.