Also, based on the analysis above, which suggests that compressive sampling incurs a log N penalty, the gap between adaptive quantization and compressive sampling will grow as the dimensions increase.įigure 4.8 is an example of three intersecting hyperplanes (three neurons) in the two-dimensional space. Any practical decoder such as orthogonal matching pursuit or lasso will do much worse. Moreover, this simulation in some sense represents the “best case” for compressive sampling since we are using an exhaustive-search decoder. While better than simple direct quantization, this performance is significantly worse than the 6 dB per bit achieved with adaptive quantization. We see that, in the very high-rate regime (greater than about 15 bits per dimension), compressive sampling with near-optimal decoding achieves an MSE reduction of approximately 4 dB per bit. This is a multiplicative rate penalty that is large by source coding standards, and it applies only at very high rates the gap is larger at lower rates. The best one could hope for with compressive sampling is that at a very high rate, M = K + 1 = 5 becomes the optimal choice, and the distortion decays as ∼2 -2( K/ M) R = 2 -2(4/5) R, or 4.8 dB per bit. This appears in Figure 5.5 as a decrease in distortion of approximately 6 dB per bit. From (5.4), the distortion with adaptive quantization decreases exponentially with the rate R through the multiplicative factor 2 -2 R. Let us now more closely compare the baseline method against compressive sampling. Scalar quantization of random measurements is not competitive with this baseline, but it provides improvement over simply coding each element of x independently (see (5.5)). The value of c used in (5.4) corresponds to scalar quantization of the nonzero entries of x. Specifically it is the convexification of (5.4) with the point ( R =0, D = 1). The “baseline” curve in Figure 5.5 shows the performance obtained by choosing judiciously-depending on the rate-between direct coding of x and adaptively coding the sparsity pattern and nonzero values. At high rates, the simulations show steeper slopes for smaller M, but note that it is not reasonable to ignore sparsity pattern detection failure recall the discussion leading to (5.12) and also see for more extensive simulations consistent with these. If ignoring the possibility of sparsity detection failure was justified, then the distortion would decay as ∼2 -2( K/ M) R. Thus, when there is more quantization noise, M must be larger to keep the probability of sparsity pattern detection failure low. Our interpretation is that, except at a very low rate, instances in which the sparsity pattern recovery fails dominate the distortion. The simulation results show that the optimal value of M decreases as the rate increases. Optimizing M trades off errors in the sparsity pattern against errors in the estimated values for the components. We reported results for lasso and orthogonal matching pursuit in. For our purposes of illustration, it also has the advantage of not requiring additional parameters (such as the λ in lasso). (The first step would be maximum likelihood detection of the sparsity pattern if y - ŷ were white Gaussian, but this is of course not the case.) This reconstruction procedure generally produces a better estimate than a convex optimization or greedy algorithm. The results reported here are for reconstructions computed by first projecting ŷ to the nearest of the K-dimensional subspaces described above and then computing the least-squares approximation assuming the sparsity pattern has been determined correctly. 4 Finding the subspace nearest to ŷ is NP-hard, and finding the P required subspaces is even harder. The optimal estimate is the conditional expectation of x given ŷ, which is the centroid of S ŷ under the Gaussian weighting of the nonzero components of x. The set of values S ŷ ⊂ ℝ N consistent with ŷ is thus the union of the P convex sets and is nonconvex whenever P > 1. Each subspace corresponds to a unique sparsity pattern for x, and the intersection between the subspace and the hypercube gives a convex set containing the nonzero components of x. This hypercube intersects one or more of the K-dimensional subspaces that contain y denote the number of subspaces intersected by P. The value of ŷ specifies an M - dimensional hypercube that contains y. Optimal estimation of x from ŷ is conceptually simple but computationally infeasible. Also plotted are the theoretical distortion curves for direct and baseline quantization. Quantization step size and number of measurements M are varied. Rate-distortion performance of compressive sampling using entropy-constrained uniform scalar quantization of random measurements and reconstruction via maximum likelihood estimation of the sparsity pattern.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |